Import Cryptech wiki dump

1 files changed, 223 insertions, 0 deletions

diff --git a/raw-wiki-dump/BuildingFromSource b/raw-wiki-dump/BuildingFromSource
new file mode 100644
index 0000000..2dc03ab
--- /dev/null
+++ b/raw-wiki-dump/BuildingFromSource
@@ -0,0 +1,223 @@
+[[PageOutline]]

+

+= Building Cryptech !Software/Firmware/Bitstream From Source =

+

+Everything you need to build our software, firmware, and FPGA

+bitstreams from source yourself is publicly available, but the process

+is a bit complicated.  Overall, there are two methods, one of which

+our developers use while writing this stuff, the other of which we use

+for the automated reproducible builds which go into our binary

+distributions.  Both methods eventually boil down to "get the source

+code then run make", but the details differ.

+

+== What developers do ==

+

+We check out copies of all the several dozen separate repositories and

+carefully arrange them in a tree structure which matches the official

+naming scheme.  Yes, really.  It's tedious, but we have

+[export:/user/sra/build-tools/https-sync-repos.py a script to automate this].

+Be warned that this script is a kludge which relies on parsing

+XML from this Wiki; this is nasty, but reasonably stable, because the

+XML itself is generated by another script.

+

+Once you have this tree, you can hop around within it, building

+whichever bits are of interest to you.  So if you want to rebuild just

+the HSM firmware (the C code that runs on the ARM), you would go to

+`sw/stm32` and run `make` there.

+

+== What we do for reproducible builds ==

+

+Reproducible builds use the same tree structure (as they must for the

+various Makefiles to work properly), but the entire tree is embedded

+in a git "superrepository" which also contains the release engineering

+goo necessary to make the whole thing work.  Do `git help submodule`

+for an introduction to git's submodule mechanism.

+

+With this model, one just checks out a copy of

+[source:/releng/alpha the superrepository],

+runs `make` in its top directory, and eventually

+the complete package pops out the other side.

+

+{{{

+git clone https://git.cryptech.is/releng/alpha.git

+cd alpha

+make

+}}}

+

+That's the good news.  The bad news is that this process has higher

+demands on its build environment: it expects to find the a complete

+tool set, including the !XiLinx synthesis tools, the several different

+cross compilers for the firmware, and the `pbuilder` system for

+building clean room packages for Ubuntu and Debian.

+

+As a compromise, one can use this source tree as if it were the

+development source tree described above: just use the supermodule to

+pull down everything else, but then ignore the supermodule and build

+individual pieces as if you'd checked out all the repositories by

+hand.

+

+== Skip all this git mess and just download a tarball ==

+

+There's another alternative, which is simpler than any of the above:

+just download the source tarball.  Since the only build environments

+we support at the moment are Debian Jessie and Ubuntu Xenial, which

+also happen to be environments for which we build binary packages, you

+can just use APT:

+

+{{{

+apt-get source cryptech-alpha

+}}}

+

+Which will give you the same tree structure, but without all the git fun.

+

+== Build environment ==

+

+Our software and firmware developers use the Debian and Ubuntu Linux

+distributions.  Our current build box for binary packages runs Debian

+Jessie.

+

+Our Verilog developers use various environments and have been known to

+use graphical tools, but synthesis of the bitstreams that go in our

+binary packages is done via the !XiLinx command line tools on the same

+Debian Jessie machine as the software and firmware builds.

+

+Which tools you need will of course depend on exactly what you're

+trying to do.

+

+Most of the tools work on either 32-bit or 64-bit machines, but if you

+intend to run the full binary package build script, you'll need a

+64-bit machine (or VM) because the tools won't build 64-bit binaries

+on a 32-bit machine.

+

+Basic tool set (not all required for every purpose, but they're all

+supported Debian packages so it's usually easier just to install them

+all and not worry about it):

+

+{{{

+  apt-get install git pbuilder ubuntu-dev-tools rsync sudo

+  apt-get install python-yaml python-serial python-crypto python-ecdsa

+  apt-get install gcc-arm-none-eabi gdb-arm-none-eabi

+  apt-get install gcc-avr binutils-avr avr-libc

+  apt-get -t jessie-backports install debootstrap distro-info-data

+  apt-get install reprepro ubuntu-archive-keyring

+}}}

+

+This is not an exhaustive list, because some of the other packages we

+use are pulled in by these as dependencies.

+

+You will also need a copy of the !XiLinx tools, which is tedious enough

+that it's described in a separate section, below.

+

+Once you have all the tools installed, you'll need a copy of the

+source tree, as explained in the preceeding sections.

+

+pbuilder requires a bit of setup (you can skip this if you're not

+trying to do the full binary package build):

+

+{{{

+for code in jessie xenial; do for arch in i386 amd64; do pbuilder-dist $code $arch create; done; done

+ln -s jessie_result ~/pbuilder/jessie-amd64_result

+ln -s xenial_result ~/pbuilder/xenial-amd64_result

+}}}

+

+== Installing the !XiLinx tools ==

+

+!XiLinx tools setup is a bit involved.  You can skip this section if

+you don't intend to build FPGA bitstreams.

+

+We use the command line versions of the !XiLinx tools, but installing

+them requires a graphical environment, because the !XiLinx installer

+and license manager are GUI tools.  If you're running this on a server

+and don't already have a graphical environment installed, you can get

+away with something fairly minimal.  For example, if you have a VNC

+viewer such as "Chicken of the VNC" on your laptop, you can get away

+with a fairly minimal X11 toolset:

+

+{{{

+apt-get install tightvncserver xterm icewm

+}}}

+

+If you're already running X11 on your laptop and are comfortable with

+extruding that to the build machine, eg, via `ssh -Y`, you can just

+use that (not recommended for long-haul use, eg, if the laptop is in

+Boston and the server is in Reykjavik).

+

+You'll need to start by using a web browser to download the

+[http://www.xilinx.com/support/download/index.html/content/xilinx/en/downloadNav/design-tools.html Xilinx ​ISE Design Suite].

+

+!XiLinx only supports specific versions of Red Hat and Suse Linux, but

+their tools do run on Debian and Ubuntu.  A few  caveats:

+

+* Debian and Ubuntu symlink `/bin/sh` to `/bin/dash`, which can't handle

+  some of the syntax used in !XiLinx's shell scripts, so you'll need to

+  change that symlink to point to `/bin/bash`.

+

+* Although the !XiLinx software can be installed as user or root, by

+  default it wants to install into /opt/Xilinx, so you need to install

+  as root if you want to do that.

+

+* The !XiLinx tools are disk hogs, so if you're building a VM for this,

+  you'll probably want to give it at least 30-40 GB of disk space.

+

+Step-by-step installation:

+

+1. Unpack `Xilinx_ISE_DS_Lin_14.7_1015_1.tar` (or whatever version you have).

+2. In an X11 environment, cd to `Xilinx_ISE_DS_Lin_14.7_1015_1`, and run `sudo ./xsetup`

+3. Click through two screens of license agreements.

+4. Select `ISE WebPACK`.

+5. Unselect (or leave unselected) Install Cable Drivers.

+6. Go!

+

+Well, not quite.  You'll need to convince the ISE tools that you have

+a valid license to use the ISE tools.  Go to

+http://www.xilinx.com/products/design-tools/ise-design-suite/ise-webpack.htm,

+click the `Licensing Solutions` link.  On the page to which that takes

+you, expand the section `Obtain a license for Free or Evaluation

+product`.  To download the ISE Webpack, you should have created an

+account, so now you can go to the Licensing Site and use that account

+to create a Certificate Based License.

+

+You do not need to go through the HostID dance, just say Do It. You

+will then receive a certificate in email (not an X.509 certificate)

+which you will be able to use. Then start the ISE Webpack by issuing

+the command `ise`.  Go to the Help menu and Manage Licenses. Use the

+resulting new License Manager window to install the `.lic` file.  This

+process is complex and flakey.

+

+Here's

+[http://www.armadeus.com/wiki/index.php?title=ISE_WebPack_installation_on_Linux another description of installing ISE on Ubuntu].

+

+The `ise` binary referred to above is in `/opt/Xilinx/14.7/ISE_DS/ISE/bin/lin64/ise`

+(or in `.../lin/ise`, but the pbuilder setup requires a 64-bit build machine).

+

+When running this remotely under tightvncserver, setup looks something like this:

+

+{{{

+vncserver :0 -geometry 1280x768 -depth 16 -localhost

+export DISPLAY=:0 XAUTHORITY=~/.Xauthority

+icewm&

+}}}

+

+Then, either in the same shell as the above or in an xterm in the new display

+

+{{{

+cd Xilinx_ISE_DS_Lin_14.7_1015_1

+sudo ./xsetup

+

+cd

+/opt/Xilinx/14.7/ISE_DS/ISE/bin/lin64/ise

+}}}

+

+It turns out you don't really need to run the whole `ise` tool to

+get to the license manager, you can just run

+

+{{{

+/opt/Xilinx/14.7/ISE_DS/common/bin/lin64/xlcm -manage

+}}}

+

+But you do have to source the appropriate settings file first, none of

+the !XiLinx tools work properly without that:

+

+{{{

+. /opt/Xilinx/14.7/ISE_DS/settings64.sh

+}}}