]> OpenEmbedded Team 2006 2007 Holger Hans Peter Freyther Koen Kooi Detlef Vollmann Jamie Lenehan Marcin Juszkiewicz This work is licensed under the Creative Commons Attribution License. To view a copy of this license, visit http://creativecommons.org/licenses/by/2.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.

Like any build tool (make, ant, jam), the OpenEmbedded build tool BitBake controls how to build things and the build dependencies. But unlike single project tools like make it is not based on one makefile or a closed set of inter-dependent makefiles, but collects and manages an open set of largely independent build descriptions (package recipes) and builds them in proper order. To be more precise: OpenEmbedded is a set of metadata used to cross-compile, package and install software packages. OpenEmbedded is being used to build and maintain a number of embedded Linux distributions, including OpenZaurus, Angström, Familiar and SlugOS. The primary use-case of OpenEmbedded are: Handle cross-compilation. Handle inter-package dependencies Must be able to emit packages (tar, rpm, ipk) Must be able to create images and feeds from packages Must be highly configurable to support many machines, distribution and architectures. Writing of metadata must be easy and reusable Together with BitBake, OpenEmbedded satisfies all these and many more. Flexibility and power have always been the priorities.

OpenEmbedded was invented and founded by the creators of the OpenZaurus project. At this time the project had pushed buildroot to its limits. It supported the creation of ipk packages, feeds and images and had support for more than one machine. But it was impossible to use different patches, files for different architectures, machines or distributions. To overcome this shortcoming OpenEmbedded was created. After a few months other projects started using OpenEmbedded and contributing back. On 7 December 2004 Chris Larson split the project into two parts: BitBake, a generic task executor and OpenEmbedded, the metadata for BitBake.

OpenEmbedded has six directories three of them hold BitBake metadata. The conf directory is holding the bitbake.conf, machine and distribution configuration. bitbake.conf is read when BitBake is started and this will include among others a local.conf the machine and distribution configuration files. These files will be searched in the BBPATH environment variable. classes is the directory holding BitBake bbclass. These classes can be inherited by the BitBake files. BitBake automatically inherits the base.bbclass on every parsed file. BBPATH is used to find the class. In packages the BitBake files are stored. For each task or application we have a directory. These directories store the real BitBake files. They are the ones ending with .bb. And for each application and version we have one.

OpenEmbedded has files ending with .conf, .inc, .bb and.bbclass. The syntax and semantic of these files are best described in the BitBake manual.

OpenEmbedded provides special BitBake classes to ease compiling, packaging and other things. FIXME.

This page will guide you trough the effort of writing a .bb file or recipe in BitBake speak. Let's start with the easy stuff, like the package description, license, etc: DESCRIPTION = "My first application, a really cool app containing lots of foo and bar" LICENSE = "GPLv2" HOMEPAGE = "http://www.host.com/foo/" The description and license fields are mandatory, so better check them twice. The next step is to specify what the package needs to build and run, the so called dependencies: DEPENDS = "gtk+" RDEPENDS = "cool-ttf-fonts" The package needs gtk+ to build ('DEPENDS') and requires the 'cool-ttf-fonts' package to run ('RDEPENDS'). OE will add run-time dependencies on libraries on its own via the so called shlibs-code, but you need to specify everything other by yourself, which in this case is the 'cool-ttf-fonts' package. After entering all this OE will know what to build before trying to build your application, but it doesn't know where to get it yet. So let's add the source location: SRC_URI = "http://www.host.com/foo/files/${P}.tar.bz2;md5sum=yoursum" This will tell the fetcher to where to download the sources from and it will check the integrity using md5sum if you provided the appropriate yoursum. You can make one by doing md5sum foo-1.9.tar.bz2 and replacing yoursum with the md5sum on your screen. A typical md5sum will look like this: a6434b0fc8a54c3dec3d6875bf3be8mtn Notice the ${P} variable, that one holds the package name, ${PN} in BitBake speak and the package version, ${PV} in BitBake speak. It's a short way of writing ${PN}-${PV}. Using this notation means you can copy the recipe when a new version is released without having to alter the contents. You do need to check if everything is still correct, because new versions mean new bugs. Before we can move to the actual building we need to find out which build system the package is using. If we're lucky, we see a configure file in the build tree this is an indicator that we can inherit autotools if we see a .pro file, it might be qmake, which needs inherit qmake. Virtually all gtk apps use autotools: inherit autotools pkgconfig We are in luck! The package is a well-behaved application using autotools and pkgconfig to configure and build it self. Lets start the build: bitbake foo Depending on what you have built before and the speed of your computer this can take a few seconds to a few hours, so be prepared. .... some time goes by ..... Your screen should now have something like this on it: NOTE: package foo-1.9-r0: task do_build: completed NOTE: package foo-1.9: completed NOTE: build 200605052219: completed All looks well, but wait, let's scroll up: NOTE: the following files where installed but not shipped: /usr/weirdpath/importantfile.foo OE has a standard list of paths which need to be included, but it can't know everything, so we have to tell OE to include that file as well: FILES_${PN} += "/usr/weirdpath/importantfile.foo" It's important to use += so it will get appended to the standard file-list, not replace the standard one.

The required version of BitBake is changing rapidly. At the time of writing (15th of October 2006) BitBake 1.6.0 was required. A safe method is to get BitBake from a stable (even mior number) Subversion branch. svn co http://svn.berlios.de/svnroot/repos/bitbake/branches/bitbake-1.6 ... A bitbake-1.6/classes/base.bbclass U bitbake-1.6 At revision 570. BitBake is checked out now; this completes the first and most critical dependency of OpenEmbedded. Issuing svn up in the bitbake-1.6 directory will update BitBake to the latest stable version, but generally it is a good idea to stick with a specific known working version of BitBake until OpenEmbedded asks you to upgrade.

The OpenEmbedded metadata has a high rate of development, so it's a good idea to stay up to date. You'll need monotone 0.28 to get the metadata and stay up to date. Monotone is available in most distributions and has binaries at Monotone homepage. Next step is getting snapshot of database. wget http://openembedded.org/snapshots/OE.mtn.bz2 http://openembedded.org/snapshots/OE.mtn.bz2.md5 Or if you have monotone 0.30 or later: wget http://www.openembedded.org/snapshots/OE-this-is-for-mtn-0.30.mtn.bz2 wget http://www.openembedded.org/snapshots/OE-this-is-for-mtn-0.30.mtn.bz2.md5 Then verify integrity of snapshot by checking md5sum. cat OE.mtn.bz2.md5sum md5sum OE.mtn.bz2 Then unpack database. bunzip OE.mtn.bz2 Finally checkout the development branch. mtn --db=OE.mtn co -b org.openembedded.dev

This section is a stub, help us by expanding it

This section is a stub, help us by expanding it

INHERIT += "debian" Placing the above line into your ${DISTRO}.conf or local.conf will trigger renaming of packages if they only ship one library. Imagine a package where the package name (PN) is foo and this packages ships a file named libfoo.so.1.2.3. Now this package will be renamed to libfoo1 to follow the Debian package naming policy.

Run-time Dependencies (RDEPENDS) will be added when packaging the software. They should only contain the minimal dependencies to run the program. OpenEmbedded will analyze each packaged binary and search for SO_NEEDED libraries. The libraries are absolutely required by the program then OpenEmbedded is searching for packages that installs these libraries. these packages are automatically added to the RDEPENDS. As a packager you don't need to worry about shared libraries anymore they will be added automatically. NOTE: This does not apply to plug-ins used by the program.

This section is a stub, help us by expanding it BBFILES := "${OEDIR}/openembedded/packages/*/*.bb ${LOCALDIR}/packages/*/*.bb" BBFILE_COLLECTIONS = "upstream local" BBFILE_PATTERN_upstream = "^${OEDIR}/openembedded/packages/" BBFILE_PATTERN_local = "^${LOCALDIR}/packages/" BBFILE_PRIORITY_upstream = "5" BBFILE_PRIORITY_local = "10"

Task-base is new way of creating basic root filesystems. Instead of having each machine setting a ton of duplicate variables, this allow a machine to specify its features and task-base builds it a customised package based on what the machine needs along with what the distro supports. To illustrate, the distro config file can say: DISTRO_FEATURES = "nfs smbfs ipsec wifi ppp alsa bluetooth ext2 irda pcmcia usbgadget usbhost" and the machine config: MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget" and the resulting task-base would support pcmcia but not usbhost. Task-base details exactly which options are either machine or distro settings (or need to be in both). Machine options are meant to reflect capabilities of the machine, distro options list things distribution maintainers might want to add or remove from their distros images.

This section is a stub, help us by expanding it

Creating a new distribution is not complicated. Config need to be created in /conf/distro directory. What has to be inside? DISTRO_VERSION so users will know which version of distribution they use. DISTRO_TYPE (release/debug) variable is used in some recipes to enable/disable some features - for example kernel output on screen for "debug" builds. Type of libc used: will it be glibc (TARGET_OS = "linux") or uclibc (TARGET_OS = "linux-uclibc")? Toolchain versions - for example gcc 3.4.4 based distro will have: PREFERRED_PROVIDERS += " virtual/${TARGET_PREFIX}gcc-initial:gcc-cross-initial" PREFERRED_PROVIDERS += " virtual/${TARGET_PREFIX}gcc:gcc-cross" PREFERRED_PROVIDERS += " virtual/${TARGET_PREFIX}g++:gcc-cross" PREFERRED_VERSION_binutils = "2.16" PREFERRED_VERSION_binutils-cross = "2.16" PREFERRED_VERSION_gcc = "3.4.4" PREFERRED_VERSION_gcc-cross = "3.4.4" PREFERRED_VERSION_gcc-initial-cross = "3.4.4" DISTRO_FEATURES which describe which features distro has. More about it in task-base section. Versions of kernels used for supported devices: PREFERRED_VERSION_linux-omap1_omap5912osk ?= "2.6.18+git" PREFERRED_VERSION_linux-openzaurus ?= "2.6.17" To get more stable build it is good to make use of sane-srcdates.inc file which contain working SRCDATE for many of floating recipes. require conf/distro/include/sane-srcdates.inc It also should have global SRCDATE value set (format is ISO date: YYYYMMDD): SRCDATE = "20061014"

To be able to build for device OpenEmbedded have to know it, so machine config file need to be written. All those configs are stored in /conf/machine/ directory. As usual some variables are required: TARGET_ARCH which describe which CPU architecture does machine use. MACHINE_FEATURES which describe which features device has. More about it in task-base section. PREFERRED_PROVIDER_virtual/kernel has to point into proper kernel recipe for this machine. Next kernel recipe needs to be added.

This section is a stub, help us by expanding it

Creating own image is easy - only few variables needs to be set: IMAGE_BASENAME to give name for own image IPKG_INSTALL to give list of packages to install into image RDEPENDS to give list of packages which are needed to be build to create image IMAGE_LINGUAS is optional list of languages which has to be installed into image Then adding of image_ipk class use: inherit image_ipk And image recipe is ready for usage.

It might be necessary to integrate a prebuilt toolchain and other libraries but still be use OpenEmbedded to build packages. One of many approaches is shown and discussed here.

We assume the toolchain provides a C and C++ compiler, an assembler and other tools to build packages. The list below shows a gcc 3.4.4 toolchain for ARM architectures using glibc. We assume that the toolchain is in your PATH. ls pre-built/cross/bin arm-linux-g++ arm-linux-ld arm-linux-ranlib arm-linux-ar arm-linux-g77 arm-linux-readelf arm-linux-as arm-linux-gcc arm-linux-gcc-3.4.4 arm-linux-c++ arm-linux-size arm-linux-c++filt arm-linux-nm arm-linux-strings arm-linux-cpp arm-linux-objcopy arm-linux-strip arm-linux-objdump

We need the header files and the libraries itself. The following directory layout is assume. PRE_BUILT has two subdirectories one is called include and holds the header files and the other directory is called lib and holds the shared and static libraries. Additionally a Qt2 directory is present having a include and lib sub-directory. ls $PRE_BUILT include lib qt2

OpenEmbedded will be setup here. We assume that your machine and distribution is not part of OpenEmbedded and they will be created ad-hoc in the local.conf file. You will need to have BitBake and a current OpenEmbedded version available.

To ease the usage of OpenEmbedded we start by creating a source-able script. This is actually a small variation from the already seen script. We will name it build_source and you will need to source it. BITBAKE_PATH=/where/is/bitbake/bin TOOLCHAIN=/where/is/toolchain/bin HOST_TOOLS=/where/is/hosttools/bin export PRE_BUILT=/where/is/pre-built export PATH=$BITBAKE_PATH:$TOOLCHAIN:$HOST_TOOLS:$PATH export OEDIR=$PWD export LOCALDIR=$PWD/secret-isv Use source build_source to source the script, use env to check that the variable where exported.

We will configure OpenEmbedded now, it is very similar to what we have done above. DL_DIR = "${OEDIR}/sources" BBFILES := "${OEDIR}/openembedded/packages/*/*.bb ${LOCALDIR}/packages/*/*.bb" BBFILE_COLLECTIONS = "upstream local" BBFILE_PATTERN_upstream = "^${OEDIR}/openembedded/packages/" BBFILE_PATTERN_local = "^${LOCALDIR}/packages/" BBFILE_PRIORITY_upstream = "5" BBFILE_PRIORITY_local = "10" BBMASK = "" ${OEDIR}/openembedded will be a upstream release of OpenEmbedded. Above we have assumed it is in the current working directory. Additionally we have a ${LOCALDIR}, we combine these two directories as a special BitBake Collection. # # machine stuff # MACHINE = "secret-killer" PACKAGE_EXTRA_ARCHS = "armv4 armv4t armv5te iwmmxt xscale"" TARGET_CC_ARCH = "-mcpu=xscale -mtune=iwmmxt" TARGET_ARCH = "arm" PACKAGE_ARCH="xscale" We tell OpenEmbedded that we build for the ARM platform and optimize for xscale and iwmmxt. INHERIT += " package_ipk debian" TARGET_OS = "linux" TARGET_FPU = "soft" DISTRO = "secret-disro" DISTRO_NAME = "secret-distro" DISTRO_VERSION = "x.y.z" DISTRO_TYPE = "release" Create a distribution ad-hoc as well. We tell OpenEmbedded that we build for linux and glibc using soft float as fpu. If your toolchain is a uclibc toolchain you will need to set TARGET_OS to linux-uclibc. export CC = "${CCACHE}arm-linux-gcc-3.4.4 ${HOST_CC_ARCH}" export CXX = "${CCACHE}arm-linux-g++ ${HOST_CC_ARCH}" export CPP = "arm-linux-gcc-3.4.4 -E" export LD = "arm-linux-ld" export AR = "arm-linux-ar" export AS = "arm-linux-as" export RANLIB = "arm-linux-ranlib" export STRIP = "arm-linux-strip" The above variables replace the ones from bitbake.conf. This will make OpenEmbedded use the prebuilt toolchain. # # point OE to the lib and include directory # TARGET_CPPFLAGS_append = " -I${PRE_BUILT}/include " TARGET_LDFLAGS_prepend = " -L${PRE_BUILT}/qt2/lib -L${PRE_BUILT}/lib \ -Wl,-rpath-link,${PRE_BUILT}/lib -Wl,-rpath-link,${PRE_BUILT}/qt2/lib " # special to Qt/Qtopia QTDIR = "${PRE_BUILT}/qt2" QPEDIR = "${PRE_BUILT}" palmtopdir = "/opt/Qtopia" palmqtdir = "/opt/Qtopia" We will add the PRE_BUILT libraries to the include and library paths. And the same is done for the special version of Qt we have in your PRE_BUILT directory. ASSUME_PROVIDED += " virtual/${TARGET_PREFIX}gcc " ASSUME_PROVIDED += " virtual/libc " ASSUME_PROVIDED += " virtual/qte " ASSUME_PROVIDED += " virtual/libqpe " ASSUME_PROVIDED += " libqpe-opie " Now we have told BitBake that the C library, compiler and Qtopia is already provided. These lines will avoid building binutils, gcc initial, glibc, gcc. source build_source bitbake your-killer-app You should be able to create the packages you want to using the prebuilt toolchain now.

If you have more prebuilt libraries you need to add additional ASSUME_PROVIDED lines to your local.conf. Using bitbake -vvv PACKAGE you can easily see the package names you could ASSUME_PROVIDED if you have some prebuilt.

NOTE: Couldn't find shared library provider for libqtopia.so.1 NOTE: Couldn't find shared library provider for libqtopia2.so.2 NOTE: Couldn't find shared library provider for libqpe.so.1 NOTE: Couldn't find shared library provider for libpthread.so.0 NOTE: Couldn't find shared library provider for libstdc++.so.6 NOTE: Couldn't find shared library provider for libqte.so.2 NOTE: Couldn't find shared library provider for libgcc_s.so.1 NOTE: Couldn't find shared library provider for libc.so.6 NOTE: Couldn't find shared library provider for libm.so.6 OpenEmbedded tries to automatically add run-time dependencies (RDEPENDS) to the package. It uses the shlibs system to do add them, in this case it was not able to find packages providing these libraries as they are prebuilt. This means they will not be added to the RDEPENDS of the just created package. The result can be fatal. If you use OpenEmbedded to create images you will end up with a image without a libc being installed. This will lead to a fatal failure. To workaround this issue you could create a package for the metadata to install every needed library and use ${BOOTSTRAP_EXTRA_RDEPENDS} to make sure this package is installed when creating images.

This section is a stub, help us by expanding it

Writing of BitBake recipes is more easy and more intuitive than writing Makefiles while providing higher flexibility. This allows you to tweak specific recipes for your very special needs and to add new recipes very fast. You can build toolchains, Software Distribution Kits (SDKs), complete Distributions or just single packages. The flexibility of OpenEmbedded allows you to reuse the once written recipes for many different purposes. OpenEmbedded provides everything buildroot will be able to provide. But in contrast to buildroot OpenEmbedded will allow you to achieve what you really want to achieve. You can add new package formats, new filesystems, new output formats easily. OpenEmbedded will suit your need.

Crosstool allows to create toolchains for you. It can only create the initial toolchain for you. It will not compile other needed libraries or applications for you, it will not be able to track dependencies or to package them properly. OpenEmbedded supports all configurations crosstool supports. You can start to create toolchains with OpenEmbedded, then as your needs grow create a more complete SDK from already present base libraries and applications and if you recognize you need to have packages for the target you have them almost built already.

Cross-compilation is a tough business. It is not that cross-compiling is hard itself but many people misuse the buildsystem they use to build their software. This will lead to a variety of issues you can run into. This can be failing tests on configuration because of executing cross compiled binaries or crashes at run-time due wrong sizes of basic types. When utilizing OpenEmbedded you avoid searching for patches at many different places and will be able to get things done more quickly. OpenEmbedded allows you to choose from a pool of ready to use software packages. OpenEmbedded will create complete flashable images using different output formats and filesystems. This allows you to create complete and specialized distributions easily.

&chapter-recipes; &class-autotools; &class-binconfig; &dirs-install; &dirs-staging; &class-distutils; &fakeroot; &class-image_ipkg; &image-types; &class-pkgconfig; &class-rootfs_ipkg; &var-section; &class-siteinfo; &var-src-uri; &class-update-alternatives; &class-update-rcd;