Arduino and GCC, compiling and uploading programs using only makefiles

Using the Arduino IDE or the Antipasto IDE can be limiting for serious development owing to their design being focused for computer newbies, abstracting away what is going on under the hood. This is especially pertinent when using the Antipasto IDE, for boards like the Liquidware Touchshield Slide, because the IDE does not by default appear to support the adding of custom .cpp and .c files to the .pde source directory, forcing the use of a single .pde file, which is ridiculous. Furthermore, both IDEs force the user to stick the PDE in a directory of the same name. Perhaps there are some workarounds for these issues, but since I'm too lazy to RTFM I decided to write a makefile to compile and upload AVR programs to boards such as the Arduino.

This example will consider the Arduino Uno since this is a popular board. It will demonstrate how to create a static library using the Arduino source, which can be called upon to compile and upload custom c and c++ programs to the Arduino Uno. It is straightforward to modify this example for other boards, including the Mega1280 and the Touchshield Slide (I have done both). This explanation is written for FreeBSD and Linux users, similar steps can probably followed on Windows using win-avr and cygwin.

The first step in building an Arduino library is to obtain the src files for the Arduino board. You can get these from git, or from your own system if you've already installed the IDE (They'll be somewhere like /usr/local/arduino/hardware/cores/arduino/src/components/board). However they are obtained, put the src files into a directory of your choice. Alternatively you can save yourself the hassle and download this complete tutorial in a gzipped tar: arduino_gcc_tutorial.tar.gz

I'm going to assume arbitrarily, that you want to put headers in ~/include/arduino and the Arduino library in ~/lib, but of course you can put these wherever you want.

Since avr-libc does not implement the C++ new and delete operators, these shall be defined by us for convenience in two additional files: forward.h and forward.cpp.

  1. forward.h
    #pragma once
    extern "C" void __cxa_pure_virtual(void);
    __extension__ typedef int __guard __attribute__((mode (__DI__)));
    extern "C" int __cxa_guard_acquire(__guard *g);
    extern "C" void __cxa_guard_release (__guard *g);
    extern "C" void __cxa_guard_abort (__guard *g);

  2. forward.cpp
    #include "forward.h"
    void __cxa_pure_virtual(void) {}
    int __cxa_guard_acquire(__guard *g) {return !*(char *)(g);};
    void __cxa_guard_release (__guard *g) {*(char *)g = 1;};
    void __cxa_guard_abort (__guard *g) {};

These should be placed in the same directory as the Arduino src so the listing looks like this:

binary.h        HardwareSerial.cpp  Print.cpp    Tone.cpp          WInterrupts.c   wiring_digital.c
wiring_pulse.c  WProgram.h          forward.cpp  HardwareSerial.h  pins_arduino.c  Print.h    
WCharacter.h    wiring_analog.c     wiring.h     wiring_shift.c    WString.cpp     forward.h
main.cpp        pins_arduino.h      Stream.h     WConstants.h      wiring.c        wiring_private.h
WMath.cpp       WString.h

Now for the Makefile. I'm using GNU make for no specific reason. On FreeBSD when installed this will be called "gmake", as BSD already has its own make. So you'll have to type this unless you alias it. Since most Linux distros use the GNU tools, it's likely that on a Linux box you can just type "make". Here is the Makefile:

CC=avr-gcc
CXX=avr-g++
MCU=-mmcu=atmega328p
CPU_SPEED=-DF_CPU=16000000UL
CFLAGS=$(MCU) $(CPU_SPEED) -Os -w
BOARD=arduino
LIBNAME=lib$(BOARD).a
INCDIR=~/include/$(BOARD)

OBJECTS=pins_arduino.o wiring.o wiring_analog.o wiring_digital.o \
        wiring_pulse.o wiring_shift.o HardwareSerial.o Print.o   \
        Tone.o WMath.o WString.o WInterrupts.o forward.o

default: $(OBJECTS)
   avr-ar rcs $(LIBNAME) $^
   mkdir -p $(INCDIR)
   mkdir -p ~/lib/
   cp *.h $(INCDIR)/
   mv $(LIBNAME) ~/lib/
   rm *.o

%.o : %.c
   $(CC) $< $(CFLAGS) -c -o $@

%.o : %.cpp
   $(CXX) $< $(CFLAGS) -c -o $@

When the make (or gmake) command is executed from the shell in the src directory the target named default is called. Now the default target depends on OBJECTS, which are all the object files required to build the Arduino library. These objects correspond to all the .c and .cpp files (except main.cpp which is no longer needed by us). Since default depends on these objects, and they do not presently exist, gmake tries to create them and looks for rules to create them. The rules which match are:

%.o : %.c
   $(CC) $< $(CFLAGS) -c -o $@

%.o : %.cpp
   $(CXX) $< $(CFLAGS) -c -o $@

These are very similar to rules implicity defined by gmake, but they have been modified slightly. They are simple rules to create object files from .c and .cpp files respectively. To take the first rule as an example, it says: in order to create a %.o target, where % is a wildcard, check for a corresponding %.c dependency and perform the action specified on the following line. In this case, the action defined invokes the avr-gcc compiler upon the automatic variable $<, which denotes the first dependency. For example, when the target wiring.o is substituted for %.o, the corresponding %.c is wiring.c. This is also the first dependency and so substitutes $< as an argument to avr-gcc.

CFLAGS specifies the compiler flags, and is defined at the top of the Makefile:

MCU=-mmcu=atmega328p
CPU_SPEED=-DF_CPU=16000000UL
CFLAGS=$(MCU) $(CPU_SPEED) -Os -w

The microcontroller (MCU) is specified with the -mmcu flag, and the CPU speed set to 16Mhz. The correct flags to use for a given Atmel MCU can be obtained by reading the manufacturers specification or by consulting Arduino's boards.txt.

The flag -Os specifies that generated code should be optimised for size, note that speed optimisations are also performed upto gcc's level 2 so long as they don't typically increase code size. The -w flag suppresses warnings (which I don't care about). The final flag which appears in the rule is -c, which tells avr-gcc not to link the object file. And finally, going back to the action line, -o specifies the desired name of the generated object file. In this case $@ is used, which is another automatic variable which matches the file name of the target of the rule, i.e the object file matched, for example wiring.o.

gmake will perform this process for all objects specified, using the second rule for C++ files, the only difference being that avr-g++ is used instead of avr-gcc. Once all the objects are created, the default rule continues as follows:

default: $(OBJECTS)
   avr-ar rcs $(LIBNAME) $^
   mkdir -p $(INCDIR)
   mkdir -p ~/lib/
   cp *.h $(INCDIR)/
   mv $(LIBNAME) ~/lib/
   rm *.o

The avr-ar line creates an archive named LIBNAME containing all the object files matched by the automatic variable $^. This automatic variable matches all of the rule dependencies, i.e the variable OBJECTS in this case. The next two instructions make the directories: ~/include/arduino and ~/lib. The forth instruction copies all the header files into ~/include/arduino so that they can be referenced later. The fifth instruction copies the generated archive libarduino.a into ~/lib so it can be used later. The final instruction deletes the generated object files which are no longer needed once the archive has been constructed. The library has now been constructed and is ready to use.

To build a library for another target board, such as the Arduino Mega only the MCU and processor speed need to be changed accordingly.

Now that the library has been constructed, it is time to demonstrate how to link it against a real program, capable of being uploaded to the Arduino. Consider the following trivial example:

#include "WProgram.h"
void setup() {
   pinMode(13,OUTPUT);
}

void loop() {
   digitalWrite(13,HIGH);
   delay(100);
   digitalWrite(13,LOW);
   delay(100);
}

int main() {
   init();
   setup();
   while(1)
      loop();
}

Notice that the usual Arduino setup() and loop() functions have been maintained for convenience, but the real work is now being done by the main function familiar to any programmers of c or C++. The header file WProgram.h is needed to reference the Arduino function such as pinMode(13,OUTPUT) or whatever. The function init is needed by wiring.c in its setup. The main function can be reused like that and so there is very little difference between a .pde and a file like this. This program flashes the LED on pin 13 like the classic blink.pde example sketch that comes with the Arduino. To compile program, link it to the library created before, and upload it to the Arduino, the following makefile can be used:

CXX=avr-gcc
INCLUDE=-I ~/include/arduino/
LIBS=-L ~/lib -lm -larduino
MCU=-mmcu=atmega328p
CPU_SPEED=-DF_CPU=16000000UL
CFLAGS=$(MCU) $(CPU_SPEED) -Os -w -Wl,--gc-sections -ffunction-sections -fdata-sections
PORT=/dev/cuaU0 # FreeBSD
ifeq ($(shell uname),Linux)
   PORT=/dev/ttyACM0
endif

default: build upload

build: Test.hex

Test.hex: Test.elf
   avr-objcopy -O ihex $< $@

OBJS= # Put other objects here
Test.elf: Test.cpp $(OBJS)
   $(CXX) $(CFLAGS) $(INCLUDE) $^ -o $@ $(LIBS)

upload:
   avrdude -V -F -p m328p -c arduino -b 115200 -Uflash:w:Test.hex -P$(PORT)

clean:
   @echo -n Cleaning ...
   $(shell rm Test.elf 2> /dev/null)
   $(shell rm Test.hex 2> /dev/null)
   $(shell rm *.o 2> /dev/null)
   @echo " done"

%.o: %.cpp
   $(CXX) $< $(CFLAGS) $(INCLUDE) -c -o $@

This time the default target depends on the build and upload targets, which means these will be called in turn. The build target depends on the file Test.hex. This is the hex file which will be uploaded to the Arduino over the USB to serial interface. This target depends on Test.elf, which is the actual gcc compiled object file. The target Test.elf, depends on Test.cpp and any other objects specified in the variable OBJECTS.

Objects specified in the OBJECTS variable will be compiled by the rule:

%.o: %.cpp
   $(CXX) $< $(CFLAGS) $(INCLUDE) -c -o $@

Which is very similar to before. Once all objects are compiled, Test.elf is compiled using the rule:

Test.elf: Test.cpp $(OBJS)
   $(CXX) $(CFLAGS) $(INCLUDE) $^ -o $@ $(LIBS)

There are a few differences to the gmake rule described previously. First all the dependencies are pulled in using the automatic variable $^ since there can be multiple inputs. And secondly the -c flag is NOT specified since it is desired to link the compiled executable to the dependent libraries. The compiler generates the compiled code and then tries to find any functions used. The places to look is specified using the flag -larduino, which causes the compiler to search for a library called libarduino.a. More generally, in this context, -lX causes the compiler to search for libX.a. The compiler searches for libarduino.a in several default locations as well as any specified using instances of the -L option, which in this case is set to ~/lib, which is the directory the library was stored in earlier. Finally, observe that the CFLAGS variable is different:

CFLAGS=$(MCU) $(CPU_SPEED) -Os -w -Wl,--gc-sections -ffunction-sections -fdata-sections

The only difference is the addition of a few new flags:

  • -Wl,--gc-sections This cuts out .o files which are not used.
  • -ffunction-sections This makes it seem to the linker essentially as if each function were in its own .o file, so that --gc-sections argument is even better.
  • -ffdata-sections This does for static and global variables what --ffunction-sections does for functions.

Once the Test.elf this rule is done, and the dependency for the Test.hex rule is satisfied, so it executes:

Test.hex: Test.elf
   avr-objcopy -O ihex $< $@

This straightforward rule invokes avr-objcopy to convert Test.elf into a format that can be recognised by the Arduino's bootloader (which accepts new programs via the USB to serial interface). The flag -o IHEX specifies the intel hexadecimal format accordingly. Note that there are several options available to avr-objcopy which are claimed to reduce hex file size, which can be useful since these boards don't have a lot of FLASH. I tried a few but it didn't seem to make any difference, so I haven't bothered with them. That's the build target finished, which means that make goes onto the upload target.

The upload target flashes the hex file Test.hex to the Arduino using avrdude:

upload:
   avrdude -V -F -p m328p -c arduino -b 115200 -Uflash:w:Test.hex -P$(PORT)

The flags are as follows

  • -V Disables automatic verify check upon uploading. Roughly halves the upload time, since the serial interface is used to re-read the uploaded program for verification.
  • -F Do not try to verify the device signature. They say the sig can get overwritten, hasn't happened to me, but we know what the device is anyway so who cares.
  • -p m328p Specifies the MCU, in this case ATmega328P, see the manpage for a list.
  • -c arduino Programmer ID, specifies the pin configuration for use in serial communication with the Arduino.
  • -b 115200 The serial baud rate (symbols per second) to use when communicating with the Arduino.
  • -Uflash:w:Test.hex This says to perform a memory operation (-U) on the flash memory of the Arduin and specifically perform a write operation (w) with the file Test.hex
  • -P$(PORT) This specifies the serial port to use to communicate with the Arduino.

Most of the values from these options I obtained from boards.txt but one could also refer to the MCU manufacturers specifications. Notice that I've also included some code to choose the port based on platform (only FreeBSD and Linux):

PORT=/dev/cuaU0 # FreeBSD
ifeq ($(shell uname),Linux)
   PORT=/dev/ttyACM0
endif

Different boards require different options, and FTDI devices can appear on a different port such as /dev/ttyUSB0. But you should easily be able to figure out how to work with any of the common boards given this framework, so I'll leave the generalisation as an exercise for the reader... (or in otherwords, I'm too lazy since I don't need it for my purposes). Incidentally, laziness gets you everywhere in programming. Whilst one cannot be so lazy so as to do nothing, having taken up the challenge, the lazy programmer takes the most efficient and direct path to completion. The lazy programmer also produces speedy code, because he cannot be bothered to wait long for his programs to start, and he hates waiting for them to finish doing their jobs.

Comments

Thanks very much! The code

Thanks very much! The code was clearly explained, which helped me a lot. Worked like a charm.

Thanks again!

Thanks for your tutorial

Thanks for your tutorial Ashley. I've been looking for how to ditch the Arduino IDE for a while. I added the Servo.* files to my library build with no problems.

Awesome.

Awesome tutorial! Keep up the

Awesome tutorial! Keep up the good work!!

Tutorial

FWIW the tutorial has 404'd again.

OK thanks, fixed. I don't

OK thanks, fixed. I don't understand why yet, must be drush or some other part of drupal, erasing the misc directory...

Couldn't find the file arduino_gcc_tutorial.tar.gz

Perhaps it has been removed from the destination url http://www.ashleymills.com/misc/arduino_gcc_tutorial.tar.gz
Your writeup was very informative - its exactly what I was looking for. One thing I need to figure out is (comparing your article to the one on http://arduino.cc/playground/FreeBSD/CLI ) is why you mention that we need the board specific files using git, but the page on the arduino site does not refer to it at all.

Thanks,

Raj
r.rajamani@gmail.com

Thanks for pointing this out.

Thanks for pointing this out. I had a comment a while back about another missing file, and now it's just dawned on me where these files have gone. I upgraded drupal a while back and must have blotted them in my haste.

I've restored it. Need to get around to doing a full backup recovery.

Concerning the link you speak of. Those dudes are not using the Arduino library, they just program the AVR directly. If you want to use the Arduino library then you need to compile and link against it, hence the need for getting the board files, in this case from git.

I guess it's an interesting question whether to program the AVR directly or use the Arduino library, since we are skipping the Arduino IDE anyway. I wouldn't have personally known where to start had the Arduino library not existed. Now that I have read bits of the library source and some other driver code I can see that it isn't a great leap of logic or difficulty to just program it directly in AVR. The choice is yours.

changes for 1.0

Thanks for updating the link, I tried your suggested makefiles and it worked like a charm. Later I tried the same approach with the arduino 1.0 code base. The minor changes are that the set of files are different and you do not need the forward.h and forward.cpp you wrote - they already have something similar (I forget the name of their file). As for using the Arduino library vs AVR files, I am using the Arduino library as that will give me less of problems along the way without having to figure out and reinvent the wheel for minor things as I go along.

Raj
r.rajamani@gmail.com

Bruno

The additional files: forward.h and forward.cpp are a excellent idea...
Thank you!

Awesome guide

Thank you very much for the detailed write up. This was remarkably perfect, and something I have looked for extensively in the past. Made everything so easy.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Fill in the blank