diff options
Diffstat (limited to 'content/BuildingFromSource.md')
| -rw-r--r-- | content/BuildingFromSource.md | 234 |
1 files changed, 234 insertions, 0 deletions
diff --git a/content/BuildingFromSource.md b/content/BuildingFromSource.md new file mode 100644 index 0000000..d39111d --- /dev/null +++ b/content/BuildingFromSource.md @@ -0,0 +1,234 @@ +Title: BuildingFromSource +Author: sra +Date: 2017-05-13 17:47 +Modified: 2017-05-17 21:28 + + + +# Building Cryptech Software/Firmware/Bitstream From Source + +Everything you need to build our software, firmware, and FPGA +bitstreams from source yourself is publicly available, but the process +is a bit complicated. Overall, there are two methods, one of which +our developers use while writing this stuff, the other of which we use +for the automated reproducible builds which go into our binary +distributions. Both methods eventually boil down to "get the source +code then run make", but the details differ. + +## What developers do + +We check out copies of all the several dozen separate repositories and +carefully arrange them in a tree structure which matches the official +naming scheme. Yes, really. It's tedious, but we have +[a script to automate this](https://git.cryptech.is/user/sra/build-tools/tree/https-sync-repos.py). +Be warned that this script is a kludge which relies on parsing +XML from this Wiki; this is nasty, but reasonably stable, because the +XML itself is generated by another script. + +Once you have this tree, you can hop around within it, building +whichever bits are of interest to you. So if you want to rebuild just +the HSM firmware (the C code that runs on the ARM), you would go to +`sw/stm32` and run `make` there. + +## What we do for reproducible builds + +Reproducible builds use the same tree structure (as they must for the +various Makefiles to work properly), but the entire tree is embedded +in a git "superrepository" which also contains the release engineering +goo necessary to make the whole thing work. Do `git help submodule` +for an introduction to git's submodule mechanism. + +With this model, one just checks out a copy of +[the superrepository](https://git.cryptech.is/releng/alpha/about), +runs `make` in its top directory, and eventually +the complete package pops out the other side. + +``` +git clone https://git.cryptech.is/releng/alpha.git +cd alpha +make +``` + +That's the good news. The bad news is that this process has higher +demands on its build environment: it expects to find the a complete +tool set, including the XiLinx synthesis tools, the several different +cross compilers for the firmware, and the `pbuilder` system for +building clean room packages for Ubuntu and Debian. + +As a compromise, one can use this source tree as if it were the +development source tree described above: just use the supermodule to +pull down everything else, but then ignore the supermodule and build +individual pieces as if you'd checked out all the repositories by +hand. + +## Skip all this git mess and just download a tarball + +There's another alternative, which is simpler than any of the above: +just download the source tarball. Since the only build environments +we support at the moment are Debian Jessie and Ubuntu Xenial, which +also happen to be environments for which we build binary packages, you +can just use APT: + +``` +apt-get source cryptech-alpha +``` + +Which will give you the same tree structure, but without all the git fun. + +## Build environment + +Our software and firmware developers use the Debian and Ubuntu Linux +distributions. Our current build box for binary packages runs Debian +Jessie. + +Our Verilog developers use various environments and have been known to +use graphical tools, but synthesis of the bitstreams that go in our +binary packages is done via the XiLinx command line tools on the same +Debian Jessie machine as the software and firmware builds. + +Which tools you need will of course depend on exactly what you're +trying to do. + +Most of the tools work on either 32-bit or 64-bit machines, but if you +intend to run the full binary package build script, you'll need a +64-bit machine (or VM) because the tools won't build 64-bit binaries +on a 32-bit machine. + +Basic tool set (not all required for every purpose, but they're all +supported Debian packages so it's usually easier just to install them +all and not worry about it): + +``` + apt-get install git pbuilder ubuntu-dev-tools rsync sudo + apt-get install python-yaml python-serial python-crypto python-ecdsa + apt-get install gcc-arm-none-eabi gdb-arm-none-eabi + apt-get install gcc-avr binutils-avr avr-libc + apt-get -t jessie-backports install debootstrap distro-info-data + apt-get install reprepro ubuntu-archive-keyring +``` + +This is not an exhaustive list, because some of the other packages we +use are pulled in by these as dependencies. + +You will also need a copy of the XiLinx tools, which is tedious enough +that it's described in a separate section, below. + +Once you have all the tools installed, you'll need a copy of the +source tree, as explained in the preceeding sections. + +pbuilder requires a bit of setup (you can skip this if you're not +trying to do the full binary package build): + +``` +for code in jessie xenial; do for arch in i386 amd64; do pbuilder-dist $code $arch create; done; done +ln -s jessie_result ~/pbuilder/jessie-amd64_result +ln -s xenial_result ~/pbuilder/xenial-amd64_result +``` + +## Installing the XiLinx tools + +XiLinx tools setup is a bit involved. You can skip this section if +you don't intend to build FPGA bitstreams. + +We use the command line versions of the XiLinx tools, but installing +them requires a graphical environment, because the XiLinx installer +and license manager are GUI tools. If you're running this on a server +and don't already have a graphical environment installed, you can get +away with something fairly minimal. For example, if you have a VNC +viewer such as "Chicken of the VNC" on your laptop, you can get away +with a fairly minimal X11 toolset: + +``` +apt-get install tightvncserver xterm icewm +``` + +If you're already running X11 on your laptop and are comfortable with +extruding that to the build machine, eg, via `ssh -Y`, you can just +use that (not recommended for long-haul use, eg, if the laptop is in +Boston and the server is in Reykjavik). + +You'll need to start by using a web browser to download the +[Xilinx ISE Design Suite](http://www.xilinx.com/support/download/index.html/content/xilinx/en/downloadNav/design-tools.html). + +XiLinx only supports specific versions of Red Hat and Suse Linux, but +their tools do run on Debian and Ubuntu. A few caveats: + + +* Debian and Ubuntu symlink `/bin/sh` to `/bin/dash`, which can't handle + some of the syntax used in XiLinx's shell scripts, so you'll need to + change that symlink to point to `/bin/bash`. + + + +* Although the XiLinx software can be installed as user or root, by + default it wants to install into /opt/Xilinx, so you need to install + as root if you want to do that. + + + +* The XiLinx tools are disk hogs, so if you're building a VM for this, + you'll probably want to give it at least 30-40 GB of disk space. + + +Step-by-step installation: + +1. Unpack `Xilinx_ISE_DS_Lin_14.7_1015_1.tar` (or whatever version you have). +2. In an X11 environment, cd to `Xilinx_ISE_DS_Lin_14.7_1015_1`, and run `sudo ./xsetup` +3. Click through two screens of license agreements. +4. Select `ISE WebPACK`. +5. Unselect (or leave unselected) Install Cable Drivers. +6. Go! + +Well, not quite. You'll need to convince the ISE tools that you have +a valid license to use the ISE tools. Go to +http://www.xilinx.com/products/design-tools/ise-design-suite/ise-webpack.htm, +click the `Licensing Solutions` link. On the page to which that takes +you, expand the section `Obtain a license for Free or Evaluation +product`. To download the ISE Webpack, you should have created an +account, so now you can go to the Licensing Site and use that account +to create a Certificate Based License. + +You do not need to go through the HostID dance, just say Do It. You +will then receive a certificate in email (not an X.509 certificate) +which you will be able to use. Then start the ISE Webpack by issuing +the command `ise`. Go to the Help menu and Manage Licenses. Use the +resulting new License Manager window to install the `.lic` file. This +process is complex and flakey. + +Here's +[another description of installing ISE on Ubuntu](http://www.armadeus.com/wiki/index.php?title=ISE_WebPack_installation_on_Linux). + +The `ise` binary referred to above is in `/opt/Xilinx/14.7/ISE_DS/ISE/bin/lin64/ise` +(or in `.../lin/ise`, but the pbuilder setup requires a 64-bit build machine). + +When running this remotely under tightvncserver, setup looks something like this: + +``` +vncserver :0 -geometry 1280x768 -depth 16 -localhost +export DISPLAY=:0 XAUTHORITY=~/.Xauthority +icewm& +``` + +Then, either in the same shell as the above or in an xterm in the new display + +``` +cd Xilinx_ISE_DS_Lin_14.7_1015_1 +sudo ./xsetup + +cd +/opt/Xilinx/14.7/ISE_DS/ISE/bin/lin64/ise +``` + +It turns out you don't really need to run the whole `ise` tool to +get to the license manager, you can just run + +``` +/opt/Xilinx/14.7/ISE_DS/common/bin/lin64/xlcm -manage +``` + +But you do have to source the appropriate settings file first, none of +the XiLinx tools work properly without that: + +``` +. /opt/Xilinx/14.7/ISE_DS/settings64.sh +``` |