From d644cb1d1be6d78e6462db631d8a01944855d1c1 Mon Sep 17 00:00:00 2001 From: Thomas Schwery Date: Fri, 18 Oct 2019 23:47:48 +0200 Subject: [PATCH] Add article on building an AppImage with Docker --- articles/2019-09-10-kicad-docker.md | 276 ++++++++++++++++++++++++++++ 1 file changed, 276 insertions(+) create mode 100644 articles/2019-09-10-kicad-docker.md diff --git a/articles/2019-09-10-kicad-docker.md b/articles/2019-09-10-kicad-docker.md new file mode 100644 index 0000000..f53c705 --- /dev/null +++ b/articles/2019-09-10-kicad-docker.md @@ -0,0 +1,276 @@ +--- +title: KiCad compilation through Docker +date: 2019-09-10 18:30:00 +--- + +While trying to compile [KiCad](http://kicad-pcb.org/download/debian/) from +the latest sources, I had problems installing the large number of required +dependencies with the correct versions on my two computers. + +I already know of a tool that allows building a system image that is in +a known state and can be executed independently of the host system : Docker. + +In this article, I will summarize the different steps I took to compile KiCad +from the sources into an AppImage that can then be copied onto the different +systems that I use for my keyboard project. + +## Packaging the dependencies +The first step is to package the dependencies needed for the compilation into a +Docker image that can then be used. The KiCad website lists the required +packages for Debian, we will thus start from this list to build our image. + +Some of the dependencies required have been updated. The final list I obtained +through trial and error was the following. + +``` +FROM debian:10 + +# Install kicad deps +RUN apt-get update -q && \ + apt-get install --no-upgrade -qqy \ + git cmake build-essential curl ccache \ + libcurl4 libcurl4-gnutls-dev \ + libboost-dev libboost-test-dev libboost-filesystem-dev libboost-regex-dev \ + liboce-foundation-dev liboce-ocaf-dev \ + ca-certificates libssl-dev \ + libngspice0-dev \ + libglew-dev libglm-dev swig \ + libcairo2-dev doxygen graphviz \ + python3-wxgtk4.0 \ + libwxgtk3.0-dev libwxgtk3.0-gtk3-dev python3 python3-dev \ + && \ + apt-get clean && \ + apt-get purge && \ + rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* +``` + +This Dockerfile will allow us to build a base system that allows us to +then manually compile the sources through Docker. The following command +can be executed from a clone KiCad repository on your computed containing +the sources. We will create a build directory from inside the container, +configure the build and compile the sources to check that everything needed +has been packaged. + +```none +host $ docker run --rm -ti -v $(pwd):/kicad 196c68cf5e05 /bin/bash +ctnr $ cd kicad/ +ctnr $ mkdir build +ctnr $ cmake \ + -DCMAKE_C_COMPILER_LAUNCHER=ccache \ + -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_PREFIX=/usr \ + -DKICAD_SCRIPTING_PYTHON3=on \ + -DKICAD_SCRIPTING_WXPYTHON_PHOENIX=on \ + ../ +ctnr $ make -j16 +``` + +You should get no error during the compilation, otherwise it means that +some dependency must be updated or installed. I followed the different errors +to build the list of dependencies given above. + +## Generating an AppImage + +[AppImage](https://appimage.org/) builds everything needed for the execution into one executable, +allowing me to copy a single file to a new computer and not having to +worry about installing any dependency and thus testing the generated +executable quickly. + +I extended the Docker image to also include the necessary tools for building +a basic AppImage : + +``` +FROM debian:10 + +# Install kicad deps +RUN apt-get update -q && \ + apt-get install --no-upgrade -qqy \ + git cmake build-essential curl ccache \ + libcurl4 libcurl4-gnutls-dev \ + libboost-dev libboost-test-dev libboost-filesystem-dev libboost-regex-dev \ + liboce-foundation-dev liboce-ocaf-dev \ + ca-certificates libssl-dev \ + libngspice0-dev \ + libglew-dev libglm-dev swig \ + libcairo2-dev doxygen graphviz \ + python3-wxgtk4.0 \ + libwxgtk3.0-dev libwxgtk3.0-gtk3-dev python3 python3-dev \ + && \ + apt-get clean && \ + apt-get purge && \ + rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* + +# Download and install linuxdeploy tool +RUN curl -O -J -L https://github.com/linuxdeploy/linuxdeploy/releases/download/continuous/linuxdeploy-x86_64.AppImage && \ + chmod +x linuxdeploy-x86_64.AppImage && \ + mv linuxdeploy-x86_64.AppImage /usr/bin/linuxdeploy && \ + curl -O -J -L https://raw.githubusercontent.com/TheAssassin/linuxdeploy-plugin-conda/master/linuxdeploy-plugin-conda.sh && \ + mv linuxdeploy-plugin-conda.sh /usr/bin/linuxdeploy-plugin-conda && \ + chmod +x /usr/bin/linuxdeploy-plugin-conda +``` + +The first `linuxdeploy` tools allows us to easily build an AppImage. The +second `linuxdeploy-plugin-conda` eases working with packaging applications +that needs Python. + +We can build the new Docker image and execute it again on the folder used +previously. As we used a volume, our previous build will still be available +allowing us to continue directly with the packaging. + +``` +host $ docker run --rm -ti -v $(pwd):/kicad cae4d304d730 /bin/bash +ctnr $ cd /kicad/build +ctrn $ make install DESTDIR=AppDir +``` + +This will install our compiled sources into an AppDir directory. We will now +need to also copy some Python dependencies that will be required and not +automatically handled : + +``` +ctnr $ mkdir -p AppDir/usr/lib/python3/dist-packages/ +ctnr $ cp -r /usr/lib/python3/dist-packages/wx \ + /usr/lib/python3/dist-packages/wxPython-4.0.4.egg-info \ + AppDir/usr/lib/python3/dist-packages/ +``` + +Then, we also need to update our KiCad binary so that the packaged Python +will be used and not the system one. For that, we will replace the KiCad +binary with a script that will simply extract the path at which the AppImage +will be mounted and export the `PYTHON_PATH` variable accordingly. + +``` +ctnr $ mv AppDir/usr/bin/kicad AppDir/usr/bin/kicad_bin +ctnr $ cat << "EOF" > AppDir/usr/bin/kicad +#!/bin/sh +HERE="$(dirname "$(readlink -f "${0}")")/../../" +export PYTHON_PATH="${HERE}"/usr/lib/python3:${PYTHON_PATH} +exec "${HERE}/usr/bin/kicad_bin" "$@" +EOF +ctnr $ chmod +x AppDir/usr/bin/kicad +``` + +Now that everything is ready, we can package the AppDir into an AppImage +using `linuxdeploy`. + +``` +ctnr $ LD_LIBRARY_PATH=$(pwd)/AppDir/usr/lib/x86_64-linux-gnu/ linuxdeploy \ + --appimage-extract-and-run \ + --appdir AppDir \ + -d $(pwd)/AppDir/usr/share/applications/kicad.desktop \ + --output appimage +``` + +This will generate a `KiCad-*-x86_64.AppImage` file in your build folder +that can be executed on your host OS without having to install any +dependency. It will not be possible to execute the AppImage from inside +the container for multiple reasons, first of which that FUSE is needed to +mount the AppImage content and is not available in a container. Second, +an X Server will be needed to start the application and is also not available +in the container. + +## Build script +Building manually without having to worry about dependencies is great but +we can do better by simply scripting the different steps so that everything +happens in one command line. + +Once the steps have been verified, we can put them in a build script +`build-docker.sh` that we include in the Docker image that will be called +when starting the container. + +``` +#!/bin/bash + +cd "$1" + +BUILD_TYPE="$2" +if [ -z "$BUILD_TYPE" ]; then + BUILD_TYPE="Debug" +fi + +mkdir build +cd build +rm -rf AppDir + +cmake \ + -DCMAKE_C_COMPILER_LAUNCHER=ccache \ + -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ + -DCMAKE_BUILD_TYPE=${BUILD_TYPE} \ + -DCMAKE_INSTALL_PREFIX=/usr \ + -DKICAD_SCRIPTING_PYTHON3=on \ + -DKICAD_SCRIPTING_WXPYTHON_PHOENIX=on \ + ../ + +make -j$(nproc) --output-sync + +if [ $? -ne 0 ]; then + exit 10 +fi + +make install DESTDIR=AppDir + +mkdir -p AppDir/usr/lib/python3/dist-packages/ + +cp -r /usr/lib/python3/dist-packages/wx \ + /usr/lib/python3/dist-packages/wxPython-4.0.4.egg-info \ + AppDir/usr/lib/python3/dist-packages/ + +mv AppDir/usr/bin/kicad AppDir/usr/bin/kicad_bin + +cat << "EOF" > AppDir/usr/bin/kicad +#!/bin/sh +HERE="$(dirname "$(readlink -f "${0}")")/../../" +export PYTHON_PATH="${HERE}"/usr/lib/python3:${PYTHON_PATH} +exec "${HERE}/usr/bin/kicad_bin" "$@" +EOF + +chmod +x AppDir/usr/bin/kicad + +LD_LIBRARY_PATH=$(pwd)/AppDir/usr/lib/x86_64-linux-gnu/ linuxdeploy \ + --appimage-extract-and-run \ + --appdir AppDir \ + -d $(pwd)/AppDir/usr/share/applications/kicad.desktop \ + --output appimage + +mv KiCad*.AppImage ../KiCad-$(git describe).AppImage +``` + +``` +FROM debian:10 + +# Install kicad deps +RUN apt-get update -q && \ + apt-get install --no-upgrade -qqy \ + git cmake build-essential curl ccache \ + libcurl4 libcurl4-gnutls-dev \ + libboost-dev libboost-test-dev libboost-filesystem-dev libboost-regex-dev \ + liboce-foundation-dev liboce-ocaf-dev \ + ca-certificates libssl-dev \ + libngspice0-dev \ + libglew-dev libglm-dev swig \ + libcairo2-dev doxygen graphviz \ + python3-wxgtk4.0 \ + libwxgtk3.0-dev libwxgtk3.0-gtk3-dev python3 python3-dev \ + && \ + apt-get clean && \ + apt-get purge && \ + rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* + +# Download and install linuxdeploy tool +RUN curl -O -J -L https://github.com/linuxdeploy/linuxdeploy/releases/download/continuous/linuxdeploy-x86_64.AppImage && \ + chmod +x linuxdeploy-x86_64.AppImage && \ + mv linuxdeploy-x86_64.AppImage /usr/bin/linuxdeploy && \ + curl -O -J -L https://raw.githubusercontent.com/TheAssassin/linuxdeploy-plugin-conda/master/linuxdeploy-plugin-conda.sh && \ + mv linuxdeploy-plugin-conda.sh /usr/bin/linuxdeploy-plugin-conda && \ + chmod +x /usr/bin/linuxdeploy-plugin-conda + +COPY build-docker.sh /build-docker.sh + +ENTRYPOINT ["/bin/bash", "/build-docker.sh"] +CMD ["/kicad"] +``` + +We can then simply build an AppImage from the latest sources with just one +command line, without having to worry about updating our host system with +the latest dependencies : `docker run --rm -it -v $(pwd):/kicad kicad-builder-docker kicad Release`