tag:blogger.com,1999:blog-15972073994088395192024-03-14T01:52:45.131-07:00Successful Unit Testtehnyithttp://www.blogger.com/profile/05019875106808614610noreply@blogger.comBlogger11125tag:blogger.com,1999:blog-1597207399408839519.post-31272550880770678282016-10-22T14:18:00.004-07:002016-10-22T14:18:45.222-07:00cmocka and cmake<div style="text-align: left;">
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I am using <code>cmocka</code> as the unit test framework to illustrate the concepts used in my book “a Guide to Success Unit Tests”.</span></div>
<div style="text-align: left;">
<br /></div>
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><div style="text-align: left;">
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><code>cmocka</code> relies on <code>cmake</code> build system to help it build the <code>cmocka</code> dynamic library as well as the test scripts into executables. So <code>cmake</code> will have to be installed as well as <code>cmocka</code> for it work correctly.</span></div>
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><code>cmake</code> can be downloaded from <code>http://www.cmake.org/download/</code></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><code>cmocka</code> can be downloaded from <code>https://cmocka.org/files/</code></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The I am using a MacOS computer, so my references are from using that computer.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><strong>Installing CMake</strong></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">When installing <code>cmake</code>, many different packaging versions
can be downloaded. Since my computer is a MacOS, I downloaded the .dmg
package. After I have extracted and installed it, I added it path the <code>cmake</code> executable.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><code>export PATH=/Applications/CMake.app/Contents/bin:$PATH</code></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I test that it can find it by getting it to report is version number.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><code>localhost:~ tehnyitchin$ ccmake --version<br />
ccmake version 3.3.0</code></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><code>CMake suite maintained and supported by Kitware (kitware.com/cmake).<br />
localhost:~ tehnyitchin$</code></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><strong>Checking cmocka is working</strong></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Once you have downloaded <code>cmocka</code> and unpacked it, you can test if the <code>cmake</code> and <code>cmocka</code> are going to work correctly. The file README has the correct instructions. In the example below, I have unzipped <code>cmocka</code> into a directory called <code>cmocka</code>.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><code><br />
cd cmocka<br />
mkdir build<br />
cd build<br />
cmake ../.<br />
make<br />
</code></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">After the build, the test binaries are in subdirectory <code>example</code>. Executing one of the test binaries will execute the test scripts. Below is a execution of <code>simple_test</code></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><code><br />
localhost:example tehnyitchin$ ./simple_test<br />
[==========] Running 1 test(s).<br />
[ RUN ] null_test_success<br />
[ OK ] null_test_success<br />
[==========] 1 test(s) run.<br />
[ PASSED ] 1 test(s).<br />
localhost:example tehnyitchin$<br />
</code></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><hr />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I am writing a <strong>Guide to Successful Unit Tests</strong>, you can get it <a href="https://leanpub.com/successfulunittest/?src=website">here at Leanpub</a></span><br />
tehnyithttp://www.blogger.com/profile/05019875106808614610noreply@blogger.com0tag:blogger.com,1999:blog-1597207399408839519.post-64212516736128841242016-10-22T14:17:00.002-07:002016-10-22T14:17:27.112-07:00Installing gcc on an oldish OSX so I can get code coverage to work<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">On one my computers that I am using, it is running OSX 10.8.5, this
is known as Mountain Lion. On this computer, I want to install the <code>gcc</code> on it, but I have some difficulties in find the right instructions.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The default <code>gcc</code> that is install is an alias to</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><code>localhost:bin tehnyitchin$ ll gcc*<br />
lrwxr-xr-x 1 root wheel 12 Aug 18 2013 gcc -> llvm-gcc-4.2</code></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">So it is not really running the real <code>gcc</code> that you can get from GNU. Scouring the internet, I came across these instructions at Helsinki University Geodynamic Group.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><a href="https://wiki.helsinki.fi/display/HUGG/Install+for+older+versions+of+Mac+OS+X">https://wiki.helsinki.fi/display/HUGG/Install+for+older+versions+of+Mac+OS+X</a></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I am working on this as I want to get code coverage working. It is not working at all with the default <code>llvm-gcc-4.2</code>. Compiling the code with a <code>--coverage</code> causes nothing to occur. </span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Typically, when the code is compiled with <code>--coverage</code>, the <code>.gcno</code> file is also generated. <code>gcov</code> uses this file as part of its code coverage measurements. If I enable the <code>-v</code> option to increase its verbosity, I can see that it is linking the <code>gcov</code> library, but I am not sure if it is actually doing it.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">This question on Stack Overflow shows that the code coverage on my OSX is not working.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><a href="http://stackoverflow.com/questions/8622293/no-code-coverage-with-mac-os-x-lion-and-xcode-4-llvm-g-4-2">http://stackoverflow.com/questions/8622293/no-code-coverage-with-mac-os-x-lion-and-xcode-4-llvm-g-4-2</a></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">As I execute a simple <code>main.c</code>, I am not getting the correct <code>.gcno</code>. However, when I switch it from <code>gcc</code> to <code>clang</code>, the <code>.gcno</code> files are generated.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Back to the story, After downloading <code>xcode-4</code> as per the
instruction from Helsinki, it failed to installed due to some unknown
reason. The logs did not show anything obvious. I did not continue with
installation as <code>clang</code> was generating the code coverage files for me. My feeling is that Apple has built <code>llvm-gcc-4.2</code> without code coverage support. To overcome this, the real <code>gcc</code> is needed. Luckily, in my case, clang came to the rescue.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span>
<hr />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I am writing a <strong>Guide to Successful Unit Tests</strong>, you can get it <a href="https://leanpub.com/successfulunittest/?src=website">here at Leanpub.</a> </span>tehnyithttp://www.blogger.com/profile/05019875106808614610noreply@blogger.com0tag:blogger.com,1999:blog-1597207399408839519.post-41430626708610656482016-10-22T14:08:00.004-07:002016-10-22T14:08:54.278-07:00Unit Test code and technical debt<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Technical debt has many different meaning. It depends on the context
that it is used in. Wikipedia has a nice statement that I quite like.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">“The debt can be thought of as work that needs to be done before a
particular job can be considered complete or proper. If the debt is not
repaid, then it will keep on accumulating interest, making it hard to
implement changes later on.” – [0]</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Unit tests is one area where technical debt can be easily obtained,
and the amount of debt can grow exceptionally quickly. It definitely
needs to be forethought and well planned before it is used in full
force. The following questions must be answered to fully.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><strong>What language will the test code be implemented in?</strong> –
This is critical contributor to technical debt. If you use a language
which the majority of your development team is not familiar with, the
chances of increasing technical debt is quite high. As the language is
not well known by your team, training will need to be provided to it and
the future team that will be maintaining the code. For example,
implementing the test code in Java while the code under test is in C
means that your development team must know Java and C to deliver good
software.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><strong>Is the infrastructure of your unit test environment well supported?</strong>
– Using a unit test framework that does not have a well supported
community (either through the online community or supported
commercially) means that the support of it must be carried out by your
development team. When the code enters maintenance phase, the unit test
framework still needs to be maintained, as specially as the computers
and operations system it is using starts to revised to newer versions.
This particularly true if the code has a long life time, as it is true
with most embedded software. As more test cases are being implemented,
the amount of technical debt gets larger and it makes supporting the
unit test framework quite difficult.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><strong>Is the unit test framework integrated well with your tool chain?</strong>
– Using a unit test framework that is not well integrated in your tool
chain means that the result from you test runs must be analysed
manually. For a small project, manual analysis of the the test result is
OK and relatively easy. However as the project gets bigger with a large
code set with high level complexity, good integration into your tool
chain is a must. This will ensure that the results are easily analysed,
and the correct notifications are generated. It will ensure continuous
integration can be achieved easily and automatically. It will also means
that removing manual work will also reduce the risk of performing the
manual process incorrectly.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">By answering these questions, there is a chance that your technical debt could become technical credit.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><hr />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I am writing a <strong>Guide to Successful Unit Tests</strong>, you can get it <a href="https://leanpub.com/successfulunittest/?src=website">here at Leanpub.</a></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><em>References</em></span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">[0] – <a href="https://en.wikipedia.org/wiki/Technical_debt">https://en.wikipedia.org/wiki/Technical_debt</a></span>tehnyithttp://www.blogger.com/profile/05019875106808614610noreply@blogger.com0tag:blogger.com,1999:blog-1597207399408839519.post-57367472971839821772016-10-22T14:07:00.003-07:002016-10-22T14:07:36.911-07:00Should it be the same or different?<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The language of the unit test framework is extremely important. It is
just as important as the language of the software module. In fact, the
test scripts that are written is a piece of code that needs to be
managed. So ideally, the principles of software engineering also applies
to the test scripts as well.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">To reduce the complexity in your test scripts, it would be ideal if
the test scripts are written in the same language as the software module
under test. If you are writing your software module in C, your test
scripts should also be written in C. If you are using rust, then rust
should be used for the test scripts.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The advantage of using the same language for both the software module
and test script, it is compiled for testing on the target. The big
assumption is that the language is already supported by your target. In
some cases, this is not. If you are developing using embedded Linux and
your test scripts are in written in python, the chance of your version
of python being supported on your target is pretty good. However, if you
are targeting an 8bit Atmel AVR, getting python support on the target
is pretty slim. If the test script is in the same language as the
software module for your target, you are pretty certain that test on
target is possible.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The other advantage of using the same language is the ability to
debug. In a mixed language environment, the chance of debugging your
test script as well as your software module is pretty small. For
example, the software module is written in C and the test script is also
written in C. When the binary is created and executed, it can be easily
debugged using <code>gdb</code> or another debugger. It can be debugged from entry into <code>main()</code> until it exits <code>main()</code>.
The variables can also be inspected and modified during debugging quite
easily. In a mixed language environment, the variables public names may
be mangled and that would make it difficult to find for inspection or
even to modified.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">To sum up, if you have no compelling reason to not keep all of your
test script and software module in the single language domain such as C,
do not mix your language.</span><br />
<hr />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I am writing a <strong>Guide to Successful Unit Tests</strong>, you can get it <a href="https://leanpub.com/successfulunittest/?src=website">here at Leanpub.</a></span><br />
tehnyithttp://www.blogger.com/profile/05019875106808614610noreply@blogger.com0tag:blogger.com,1999:blog-1597207399408839519.post-86231416995144086332016-10-22T14:06:00.001-07:002016-10-22T14:06:12.531-07:00Write the code first, test code that is.<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">In a software development process, certain steps must be taken to
ensure that your software module is delivered as bug free as possible.
The aim is for it to be working without any bugs. However, the question
is what bug free is exactly. One possible definition is the software
module is consider bug free if it fulfills all its requirements,
functional and non-functional. Beyond that definition, it is hard to
control the injection of bugs from outer sources. For example, if the
compiler is not generating the code and causes a mis-behaviour of the
software module. Technically, this is a bug for the product and will
still to be fixed, but it is hard to control if the behaviour of the
compiler is not well known.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">So with that definition, the steps in the process are clear. First,
the requirements are finalised, and then the implementation can start.
Since the requirements are finalised, it would make sense for the test
code to be written first. From the requirements, the functional
behaviour of the software module is well defined. This means that the
input data into the software module is known as well as the results from
it. If these are not clear or have some doubts, it is a sign that the
requirements are not clear enough. The requirements need more work.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Once the test code are written and executed. Obviously, all the test
cases will fail as the code for the software module is not written yet.
Now is the write time to start writing your code for the software
module. As you continue to write the software module, the test cases are
used as a validation for the software module. The implementation
continues until all your test cases passes.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Suppose the code for software module is written before the test code.
The downside is that the chance of getting your software module correct
is low and you won’t know it until you have some test code to do the
testing. You have just invested into some technical debt. If your test
code is written first, it is very likely to be correct the first time as
all the inputs, all the outputs and the behaviours are well defined.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Save time and effort by writing the test code first, and then write the code for your software module.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><hr />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I am writing a <strong>Guide to Successful Unit Tests</strong>.<br />
You can get it <a href="https://leanpub.com/successfulunittest/?src=website">here at Leanpub.</a> or <a href="https://gum.co/vDwq?src=website">here at Gumroad.</a> and read about these topics and more.</span>tehnyithttp://www.blogger.com/profile/05019875106808614610noreply@blogger.com0tag:blogger.com,1999:blog-1597207399408839519.post-72494281631018853102016-10-22T13:59:00.004-07:002016-10-22T13:59:53.368-07:00Where are your input values coming from?<div class="entry-content">
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">When design your test cases, you will have to decide what type of
input values are want to use, but consider where your input values are
coming form. The most obvious way of inject values are by passing into
via the function parameters. It is logical and it is the formal method.
However, that is only one of the ways of injecting values into it. The
three most popular methods are.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><strong>Function Parameters</strong> -Function parameters are part of
full definitions of the function. It is one of the fundamental ways of
passing data into the function. The range of values that it can take are
defined by its type. Function Parameters can also be used to get values
out of the function. In C, the function parameters live on the stack
memory, and its life time ends when the function ends.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><strong>Returned Values</strong> – The low level functions called by
the your module under tests must be considered as an input. From a unit
test perspective, the return values can be influenced via the mocked
function. The range of values is dependent on its data type. In C, the
function parameters live on the stack memory, and its life time ends
when the function ends. The other thing to consider is that this method
does not necessary inject the values at the start of your function under
test.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><strong>Global Variables</strong> – variables used in the software
modules that are accessible outside of the modules. This method is best
avoided in complicated code bases as it tends to create a software
module that is prone to unpredictable behaviour if the global variables
are not controlled well. The global variable lives on the heap, and it
is alive as long as the program is alive.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">For most parts, the input methods comes down the how you have design
your module. Some design guideline forbid the use of mechanism such as
global variables, so make sure that your methods matches your design
guidelines.</span><br />
<hr />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I am writing a <strong>Guide to Successful Unit Tests</strong>.<br />
You can get it <a href="https://leanpub.com/successfulunittest/?src=website">here at Leanpub.</a> or <a href="https://gum.co/vDwq?src=website">here at Gumroad.</a> and read about these topics and more.</span><br />
</div>
tehnyithttp://www.blogger.com/profile/05019875106808614610noreply@blogger.com0tag:blogger.com,1999:blog-1597207399408839519.post-88261112153242574052016-10-22T13:58:00.003-07:002016-10-22T13:58:51.509-07:00Rust approach to tests<header class="entry-header"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span class="sep"></span><a href="http://www.successfulunittest.com/2016/02/03/rust-approach-to-tests/" rel="bookmark" title="9:58 pm"><time class="entry-date" datetime="2016-02-03T21:58:06+00:00"></time></a></span>
<div class="comments-link">
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><a href="http://www.successfulunittest.com/2016/02/03/rust-approach-to-tests/#respond"><span class="leave-reply"></span></a></span> </div>
</header><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><div class="entry-content">
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">About nine months ago, I started to hear good things about a new
language called Rust. It really gain traction for me when I read a blog
post by Gregely Imreh (<a href="https://twitter.com/imrehg">@imrehg</a>) who wrote about it as his <a href="https://gergely.imreh.net/blog/2015/11/language-of-the-month-rust/">language of the month</a>.
The two things that I read about Rust are that it is fast and it is
safe. Both designed from the outset with these two items in mind. For a
system level language, this is great!</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I started reading the <a href="https://doc.rust-lang.org/book/">rust docs</a>
and was pleasantly surprise that unit test is a built in design feature
of language. I was really surprise at this. The thing with most
languages, unit test is an add on feature that is implemented by another
vendor, or another group. Look at C, apart from the fact that its
behavior is sometimes unpredictable, a unit test framework is not even
defined. You will have to use <a href="http://www.successfulunittest.com/2015/09/19/cmocka-and-cmake/">cmocka</a>, or <a href="http://www.throwtheswitch.org/unity">Unity</a> or one of the other many C test frameworks. Same with python, php, Java etc.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">All the details about testing on Rust are show in this <a href="https://doc.rust-lang.org/book/testing.html">chapter in this book</a>.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The more I read about Rust, it strikes me the designers are seasoned C
language developers, suffered through many projects with the deficiency
of C, and setup to design a language with those deficiency out of the
language. Rust is looking really good as a replacement for C or C++.</span><br />
<hr />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I am writing a <strong>Guide to Successful Unit Tests</strong>.<br />
You can get it <a href="https://leanpub.com/successfulunittest/?src=website">here at Leanpub.</a> or <a href="https://gum.co/vDwq?src=website">here at Gumroad.</a> and read about these topics and more.</span><br />
</div>
tehnyithttp://www.blogger.com/profile/05019875106808614610noreply@blogger.com0tag:blogger.com,1999:blog-1597207399408839519.post-69015084640730718522016-10-22T13:56:00.003-07:002016-10-22T13:57:29.934-07:00Blinking LEDs<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">In the embedded world, blinking a LED is the equivalent to the “Hello
World” program. The simple task of blinking a LED goes a long way to
proving that you have work flow that works. It is a huge milestone for
the project. This activity is sometimes known as bringing up the board.
It is when a new board is provided and software is executed on it.</span><br />
<br />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">
</span><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">For those who are not familiar with the embedded concept, blinking a
LED proves that your tool chain is good, your understanding of accessing
the hardware is sound as well.</span><br />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">
</span><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">The most simple implementation of the blinking led application is</span><br />
<br />
<code class="cpp color1 bold">int</code> <code class="cpp plain">main(</code><code class="cpp keyword bold">void</code><code class="cpp plain">)</code><br />
<div class="container">
<div class="line number2 index1 alt1">
<code class="cpp plain">{</code></div>
<div class="line number3 index2 alt2">
<code class="cpp spaces"> </code><code class="cpp plain">PORTA.0.cfg = PUSH_PULL_OUTPUT;</code></div>
<div class="line number4 index3 alt1">
</div>
<div class="line number5 index4 alt2">
<code class="cpp spaces"> </code><code class="cpp keyword bold">while</code> <code class="cpp plain">(1)</code></div>
<div class="line number6 index5 alt1">
<code class="cpp spaces"> </code><code class="cpp plain">{</code></div>
<div class="line number7 index6 alt2">
<code class="cpp spaces"> </code><code class="cpp plain">PORTA.0.data ^= 0X01;</code></div>
<div class="line number8 index7 alt1">
<code class="cpp spaces"> </code><code class="cpp plain">}</code></div>
<div class="line number9 index8 alt2">
</div>
<div class="line number10 index9 alt1">
<code class="cpp spaces"> </code><code class="cpp keyword bold">return</code> <code class="cpp plain">0;</code></div>
<div class="line number11 index10 alt2">
<code class="cpp plain">}</code></div>
</div>
<br />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">The code will exclusive ORed the current value on PORTA.0,
effectively flipping the bit. If are able to build this code and
programmed it into your board, you can put a CRO on pin PORTA.0 and see a
nice signal toggling up and down. The frequency and duty cycle are
defaulted to the fastest setting possible. </span><br />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">
</span>
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">If you able to see a toggling signal, you have reached a significant
milestone. The hardware is considered good enough to start developing
code upon. Lets break it down.</span><br />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">
</span>
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;"><b>Your compiler and linker setup.</b> By seeing the
toggling signal on PORTA.0, it means that your tool chain has created
binary code that your microcontroller can understand. It also means that
your tool chain has linked all the binary, resolved the addresses in
your code correctly, and that your specified memory map allows the
microcontroller to boot. Getting the memory map is a tricky task as you
have address locations and sizes correct for items such as stacks, heap,
constant data etc.</span><br />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">
</span>
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;"><b>Your debugging setup.</b> By seeing the toggling signal,
it shows that you can program the microcontroller with your debugger
setup and trigger the code. If your debugger setup to the
microcontroller is via JTAG, you can set break points and inspect
variables and memory addresses. This is a big confidence booster to the
developers as it provides them the tools to work on the problem.</span><br />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">
</span>
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;"><b>Your hardware.</b> By seeing the toggling signal, it
shows that your hardware platform is powered correctly and is able to
provide you with a stable platform to start developing code.</span><br />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">
</span>
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">If you are wondering if the above code will actually blink a led.
Well, unless the frequency of your microcontroller is quite low, it will
probably not illuminate a LED. The frequency of the output is probably
way too high. To have it blink, replace the <code>while(1)</code> loop
with code that is tied to a timer event that occurs every 500ms. This
will prove the hardware further. You will also have to check if the
output pin can drive enough current it to the LED for illumination.
Otherwise, you will have to drive transistor or a FET to drive the LED. </span><br />
<br />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">
</span><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">One critical thing to remember when bringing up your board; the
project is still very young. Even though you have proved some parts of
your development flow, tool chain and development hardware, the rest of
pieces of the puzzle are yet to be solved. However, it is a big step
just to be able to blink a led. </span><br />
<hr />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">
</span><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">I am writing a <b>Guide to Successful Unit Tests</b>.<br />
You can get it <a href="https://leanpub.com/successfulunittest/?src=website">here at Leanpub.</a> or <a href="https://gum.co/vDwq?src=website">here at Gumroad.</a> and read about these topics and more.</span>tehnyithttp://www.blogger.com/profile/05019875106808614610noreply@blogger.com0tag:blogger.com,1999:blog-1597207399408839519.post-62261835341380991652016-10-22T13:54:00.000-07:002016-10-22T13:54:07.512-07:00Can a unit test replace a debugger?<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I started working in embedded systems in 1992. At that time, I have
wondered if it is possible to get your code working without the need of a
debugger.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The debuggers I was using when I started was ICE [0]. By this I meant
that the microcontroller was replaced with a special bonded out chip,
and its clocking was controlled. The ICE allow me full access to
microcontroller’s operation and peripherals, almost down the controlling
each clock cycle. I was able to see everything the microcontroller was
able to see. Sometimes, the cache on the ICE was large enough that I was
also able capture a log of the instruction sequences. This is very
helpful when I had to double check the execution part leading up to a
beak point.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">These days I work in a large engineering team with 32bit devices. The
debuggers are usually interfaced via the JTAG connector. Although I can
use it to debug my code, the control is not as fine as that of an ICE.
The direction seems to be heading towards higher level of abstraction.
Only use a debugger when it is necessary to. With a typical layered
architecture, this is workable.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Extrapolating it further, is it possible to do a full project without
any access to a debugger? In many projects, these is already occurring.
The large amount of work from the ESP8266 [1] scene, the majority of
the code are developed without a debugger. Yet this is in the hobbyist
or prototype domain. What about the professional domain? To remove the
debugger from the development tool chain would be difficult, especially
if I am debugging code that is close to the metal. However, the higher
up the software stack my code is residing in, reliance on it is getting
smaller.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Using a unit test frame, the main advantage is to de-couple its
dependency from the hardware and its maturity level. It may be possible
for the software to be completed and released before the hardware is
manufactured. Provided the behavior of the hardware is well known, this
is a good strategy for reducing development time.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">For the debugger, its major strength is to allow the code to be
debugged close to the microcontroller. If I need to double check any
data from the hardware, a debugger is essential. This is one area a unit
test framework can never accomplish. The hardware may be mocked, but
the mocked behavior will need to be verified against the hardware. </span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">[0] – In circuit emulator. <a href="http://www.ganssle.com/articles/BegincornerICE.htm">http://www.ganssle.com/articles/BegincornerICE.htm</a><br />
[1] – <a href="https://en.wikipedia.org/wiki/ESP8266">https://en.wikipedia.org/wiki/ESP8266</a></span><br />
<hr />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I am writing a <strong>Guide to Successful Unit Tests</strong>.<br />
You can get it <a href="https://leanpub.com/successfulunittest/?src=website">here at Leanpub.</a> or <a href="https://gum.co/vDwq?src=website">here at Gumroad.</a> and read about these topics and more.</span><br />
tehnyithttp://www.blogger.com/profile/05019875106808614610noreply@blogger.com0tag:blogger.com,1999:blog-1597207399408839519.post-59124927679549830072016-10-22T13:51:00.002-07:002016-10-22T13:51:49.804-07:00Why you need to measure test coverage?<div class="entry-content">
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I was at a meetup recently and I asked the question on what tools
are available for measuring coverage data during a test run for an up
and coming programming language. Some comments were made that coverage
measurements are not that important in the scope of unit testing. I was
rather taken back by this comment. Coverage measurements must be part of
the data that is used how determine how successful the testing was. </span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">To get useful measurement of coverage, lets go through what is meant
by coverage. From a testing perspective, if a requirement can be
validated by tests, the requirement is covered by tests. The validation
comes by testing the implementation’s behaviour is the same as the
requirement.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><strong>Stuff that needs coverage </strong></span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><strong>Executed code</strong> – There are code that you write that
are absolutely critical to the functionality of your product. Software
modules such as your scheduler or startup code must be covered. These
modules are core. There are also code that will be executed but its
functionality are not so critical. An example might be a module that
performs a maths calculation, its accuracy is important but the time it
takes is not so important. Both types of code must be unit tested as
they are executed.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><strong>Safety Critical Code </strong>– These code performs functions
that are critical to the safety of the user. This might include
software function dealing with air bag system in a car. These type of
code must be covered to ensure its correct behavior.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><strong>Code that will be audited </strong>– Some of the code written
will be audited to ensure that it conforms to a set of design
principles. The validation of these code are based on the auditing. </span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><strong>Safe path execution</strong> – not only should the code
execution be covered, the path which the execution takes should also be
covered. For code dealing with functional safety, the safe execution
path must be mapped out when a failure occurs. In this context, failure
means a failure that can be detected and managed. For example, the code
detects that some critical memory has become corrupted, so the execution
path to recover or to contain the corruption must be checked.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I have only given four examples where coverage is a very good tool to
use. It is the only the metric to ensure that all of your code is
tested.</span><br />
<br />
<hr />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I am writing a <strong>Guide to Successful Unit Tests</strong>.<br />
You can get it <a href="https://leanpub.com/successfulunittest/?src=website">here at Leanpub.</a> or <a href="https://gum.co/vDwq?src=website">here at Gumroad.</a> and read about these topics and more.</span><br />
</div>
tehnyithttp://www.blogger.com/profile/05019875106808614610noreply@blogger.com0tag:blogger.com,1999:blog-1597207399408839519.post-43067311657183250342016-10-22T13:34:00.006-07:002016-10-22T13:34:56.374-07:00Making sure that your unit testing is painless to run.<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">To reduce the obstacle of using unit tests in your development flow,
it is a good idea to integrate it as part of your development work flow.
The lower the barrier to setting it up, the more likely unit testing is
going to be used.</span><br />
<br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The classical development flow is to have your design specifications
stabilised and ready for implementation. In reality, at this early
stage, stabilisation has different meaning to when the software module
is ready for release. The importance of having the public functions
stabilised is critical here. There are two reasons why.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The first one is that the users of your functions can rely on those
published function signatures and have an understand of its behaviour.
This is obvious and is one of the tenets of modern software development.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The second is if your unit testers are not the actual developers, the
unit testers can start working on the test code once the public
functions are known. There will be some adjustments to function calls as
your module is being used.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">However, the main key for using unit test in your development flow is to run the test with minimal effort.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">When you are doing a build of your module, also include a step in
your build process to execute the tests. In your build process, create a
dependency of unit test runnable based on your source code and unit
test code. As soon as any of those changes, your unit test will get
built and executed.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Any mismatch between the unit test code and your module will get
picked up. By having your unit test code written by another person, it
also creates a method of validating the design documentation. It checks
that both code and documentation are aligned.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">By making the execution as one of the steps in your build process, it
is essential to make a decision whether you want to do when the unit
test fails. There are two obvious ways.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The most obvious ways is to treat the unit test fails as warnings and
allow the build to complete. I am not a big fan of this method as it
puts a lesser importance of unit tests.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">The more discipline way is to treat unit test fails as build errors
and stop the build process from moving onto the next stop until the
errors are fixed. This is much stricter method and is a more discipline
method of software development.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Whatever build system you are using, spend some time to integrate the
building and executing of unit tests into your build sequences. It will
be worth the effort in the long run.</span><br />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><hr />
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</span><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">I have written a <strong>Guide to Successful Unit Tests</strong>.<br />
You can get it <a href="https://leanpub.com/successfulunittest/?src=website">here at Leanpub.</a> or <a href="https://gum.co/vDwq?src=website">here at Gumroad.</a> and read about these topics and more.</span>tehnyithttp://www.blogger.com/profile/05019875106808614610noreply@blogger.com0