summaryrefslogtreecommitdiff
path: root/raw-wiki-dump/BuildingFromSource
diff options
context:
space:
mode:
authorRob Austein <sra@hactrn.net>2020-09-13 23:04:30 +0000
committerRob Austein <sra@hactrn.net>2020-09-13 23:04:30 +0000
commitb092ffbcbe2c9398494f7dc9db6f0796971633e0 (patch)
tree6fabf690f1ebf485a9fea9af5298e44ad2a59a3e /raw-wiki-dump/BuildingFromSource
parent9d927e49d9c10fc16c6dfa4a2a96cdb6216e4e2b (diff)
Import Cryptech wiki dump
Diffstat (limited to 'raw-wiki-dump/BuildingFromSource')
-rw-r--r--raw-wiki-dump/BuildingFromSource223
1 files changed, 223 insertions, 0 deletions
diff --git a/raw-wiki-dump/BuildingFromSource b/raw-wiki-dump/BuildingFromSource
new file mode 100644
index 0000000..2dc03ab
--- /dev/null
+++ b/raw-wiki-dump/BuildingFromSource
@@ -0,0 +1,223 @@
+[[PageOutline]]
+
+= Building Cryptech !Software/Firmware/Bitstream From Source =
+
+Everything you need to build our software, firmware, and FPGA
+bitstreams from source yourself is publicly available, but the process
+is a bit complicated. Overall, there are two methods, one of which
+our developers use while writing this stuff, the other of which we use
+for the automated reproducible builds which go into our binary
+distributions. Both methods eventually boil down to "get the source
+code then run make", but the details differ.
+
+== What developers do ==
+
+We check out copies of all the several dozen separate repositories and
+carefully arrange them in a tree structure which matches the official
+naming scheme. Yes, really. It's tedious, but we have
+[export:/user/sra/build-tools/https-sync-repos.py a script to automate this].
+Be warned that this script is a kludge which relies on parsing
+XML from this Wiki; this is nasty, but reasonably stable, because the
+XML itself is generated by another script.
+
+Once you have this tree, you can hop around within it, building
+whichever bits are of interest to you. So if you want to rebuild just
+the HSM firmware (the C code that runs on the ARM), you would go to
+`sw/stm32` and run `make` there.
+
+== What we do for reproducible builds ==
+
+Reproducible builds use the same tree structure (as they must for the
+various Makefiles to work properly), but the entire tree is embedded
+in a git "superrepository" which also contains the release engineering
+goo necessary to make the whole thing work. Do `git help submodule`
+for an introduction to git's submodule mechanism.
+
+With this model, one just checks out a copy of
+[source:/releng/alpha the superrepository],
+runs `make` in its top directory, and eventually
+the complete package pops out the other side.
+
+{{{
+git clone https://git.cryptech.is/releng/alpha.git
+cd alpha
+make
+}}}
+
+That's the good news. The bad news is that this process has higher
+demands on its build environment: it expects to find the a complete
+tool set, including the !XiLinx synthesis tools, the several different
+cross compilers for the firmware, and the `pbuilder` system for
+building clean room packages for Ubuntu and Debian.
+
+As a compromise, one can use this source tree as if it were the
+development source tree described above: just use the supermodule to
+pull down everything else, but then ignore the supermodule and build
+individual pieces as if you'd checked out all the repositories by
+hand.
+
+== Skip all this git mess and just download a tarball ==
+
+There's another alternative, which is simpler than any of the above:
+just download the source tarball. Since the only build environments
+we support at the moment are Debian Jessie and Ubuntu Xenial, which
+also happen to be environments for which we build binary packages, you
+can just use APT:
+
+{{{
+apt-get source cryptech-alpha
+}}}
+
+Which will give you the same tree structure, but without all the git fun.
+
+== Build environment ==
+
+Our software and firmware developers use the Debian and Ubuntu Linux
+distributions. Our current build box for binary packages runs Debian
+Jessie.
+
+Our Verilog developers use various environments and have been known to
+use graphical tools, but synthesis of the bitstreams that go in our
+binary packages is done via the !XiLinx command line tools on the same
+Debian Jessie machine as the software and firmware builds.
+
+Which tools you need will of course depend on exactly what you're
+trying to do.
+
+Most of the tools work on either 32-bit or 64-bit machines, but if you
+intend to run the full binary package build script, you'll need a
+64-bit machine (or VM) because the tools won't build 64-bit binaries
+on a 32-bit machine.
+
+Basic tool set (not all required for every purpose, but they're all
+supported Debian packages so it's usually easier just to install them
+all and not worry about it):
+
+{{{
+ apt-get install git pbuilder ubuntu-dev-tools rsync sudo
+ apt-get install python-yaml python-serial python-crypto python-ecdsa
+ apt-get install gcc-arm-none-eabi gdb-arm-none-eabi
+ apt-get install gcc-avr binutils-avr avr-libc
+ apt-get -t jessie-backports install debootstrap distro-info-data
+ apt-get install reprepro ubuntu-archive-keyring
+}}}
+
+This is not an exhaustive list, because some of the other packages we
+use are pulled in by these as dependencies.
+
+You will also need a copy of the !XiLinx tools, which is tedious enough
+that it's described in a separate section, below.
+
+Once you have all the tools installed, you'll need a copy of the
+source tree, as explained in the preceeding sections.
+
+pbuilder requires a bit of setup (you can skip this if you're not
+trying to do the full binary package build):
+
+{{{
+for code in jessie xenial; do for arch in i386 amd64; do pbuilder-dist $code $arch create; done; done
+ln -s jessie_result ~/pbuilder/jessie-amd64_result
+ln -s xenial_result ~/pbuilder/xenial-amd64_result
+}}}
+
+== Installing the !XiLinx tools ==
+
+!XiLinx tools setup is a bit involved. You can skip this section if
+you don't intend to build FPGA bitstreams.
+
+We use the command line versions of the !XiLinx tools, but installing
+them requires a graphical environment, because the !XiLinx installer
+and license manager are GUI tools. If you're running this on a server
+and don't already have a graphical environment installed, you can get
+away with something fairly minimal. For example, if you have a VNC
+viewer such as "Chicken of the VNC" on your laptop, you can get away
+with a fairly minimal X11 toolset:
+
+{{{
+apt-get install tightvncserver xterm icewm
+}}}
+
+If you're already running X11 on your laptop and are comfortable with
+extruding that to the build machine, eg, via `ssh -Y`, you can just
+use that (not recommended for long-haul use, eg, if the laptop is in
+Boston and the server is in Reykjavik).
+
+You'll need to start by using a web browser to download the
+[http://www.xilinx.com/support/download/index.html/content/xilinx/en/downloadNav/design-tools.html Xilinx ​ISE Design Suite].
+
+!XiLinx only supports specific versions of Red Hat and Suse Linux, but
+their tools do run on Debian and Ubuntu. A few caveats:
+
+* Debian and Ubuntu symlink `/bin/sh` to `/bin/dash`, which can't handle
+ some of the syntax used in !XiLinx's shell scripts, so you'll need to
+ change that symlink to point to `/bin/bash`.
+
+* Although the !XiLinx software can be installed as user or root, by
+ default it wants to install into /opt/Xilinx, so you need to install
+ as root if you want to do that.
+
+* The !XiLinx tools are disk hogs, so if you're building a VM for this,
+ you'll probably want to give it at least 30-40 GB of disk space.
+
+Step-by-step installation:
+
+1. Unpack `Xilinx_ISE_DS_Lin_14.7_1015_1.tar` (or whatever version you have).
+2. In an X11 environment, cd to `Xilinx_ISE_DS_Lin_14.7_1015_1`, and run `sudo ./xsetup`
+3. Click through two screens of license agreements.
+4. Select `ISE WebPACK`.
+5. Unselect (or leave unselected) Install Cable Drivers.
+6. Go!
+
+Well, not quite. You'll need to convince the ISE tools that you have
+a valid license to use the ISE tools. Go to
+http://www.xilinx.com/products/design-tools/ise-design-suite/ise-webpack.htm,
+click the `Licensing Solutions` link. On the page to which that takes
+you, expand the section `Obtain a license for Free or Evaluation
+product`. To download the ISE Webpack, you should have created an
+account, so now you can go to the Licensing Site and use that account
+to create a Certificate Based License.
+
+You do not need to go through the HostID dance, just say Do It. You
+will then receive a certificate in email (not an X.509 certificate)
+which you will be able to use. Then start the ISE Webpack by issuing
+the command `ise`. Go to the Help menu and Manage Licenses. Use the
+resulting new License Manager window to install the `.lic` file. This
+process is complex and flakey.
+
+Here's
+[http://www.armadeus.com/wiki/index.php?title=ISE_WebPack_installation_on_Linux another description of installing ISE on Ubuntu].
+
+The `ise` binary referred to above is in `/opt/Xilinx/14.7/ISE_DS/ISE/bin/lin64/ise`
+(or in `.../lin/ise`, but the pbuilder setup requires a 64-bit build machine).
+
+When running this remotely under tightvncserver, setup looks something like this:
+
+{{{
+vncserver :0 -geometry 1280x768 -depth 16 -localhost
+export DISPLAY=:0 XAUTHORITY=~/.Xauthority
+icewm&
+}}}
+
+Then, either in the same shell as the above or in an xterm in the new display
+
+{{{
+cd Xilinx_ISE_DS_Lin_14.7_1015_1
+sudo ./xsetup
+
+cd
+/opt/Xilinx/14.7/ISE_DS/ISE/bin/lin64/ise
+}}}
+
+It turns out you don't really need to run the whole `ise` tool to
+get to the license manager, you can just run
+
+{{{
+/opt/Xilinx/14.7/ISE_DS/common/bin/lin64/xlcm -manage
+}}}
+
+But you do have to source the appropriate settings file first, none of
+the !XiLinx tools work properly without that:
+
+{{{
+. /opt/Xilinx/14.7/ISE_DS/settings64.sh
+}}}