From 76f6dba6d19039d5634442a8731e4aba17a3330b Mon Sep 17 00:00:00 2001 From: Rob Austein Date: Wed, 29 Jun 2016 21:33:09 -0400 Subject: Document the build process. --- README.md | 156 ++++++++++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 110 insertions(+), 46 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index e3f3eff..cca2619 100644 --- a/README.md +++ b/README.md @@ -1,66 +1,130 @@ Alpha Releng ============ -Release engineering stuff for Cryptech Alpha. +Release engineering stuff for Cryptech Alpha board and support software. -Work in progress, currently a merge in progress of two separate -release engineering repositories, one for HSM firmware, one for host -software. -This README is probably obsolete by the time you're reading it. +## Overview ## -## Old Firmware README ## +The Makefiles and scripts in this repository attempt to automate the +process of building packaged versions of the firmware and software for +the Cryptech Alpha board. At a high level, this consists of two main +phases: -Preliminary release engineering super-repository for building firmware -for the Cryptech "Alpha" board. +1. Building a firmware package (Verilog and C code which runs on the + Alpha board itself, regardless the host system to which the Alpha + is connected); and -Primary tasks here are to build a bitstream for the FPGA and the -"bootstrap" and "hsm" images for the Alpha's CPU. +2. Building software packages (code which runs on the host to which + the Alpha is connected). -Eventually there will be a lot of packaging and versioning glorp here, -but let's start with basic build and clean targets. +Since the firmware is the same for any given version of the Cryptech +source code, and since the process of building the firmware binaries +is both time-consuming and involves tools like the XiLinx Verilog +synthesis toolkit and cross-compilers for multiple CPUs (the Alpha's +Cortex M4 ARM CPU and the Atmel AVR ATtiny828 MCU used to implement +the anti-tampering logic for the master key memory), we include a +signed tarball containing pre-built binaries for all the firmware as +part of the source tarball for the software packages. -Current repository structure is, um, complicated. On the RTL side, we -have a tree of simple subrepositories, each representing one RTL core. +You are of course free to build all of this for yourself, with or +without modification, the pre-built versions are just a convenience. -On the software side, we have a subrepository which has several -subrepositories of its own: current thinking is that this should -probably be replaced by separate repositories and Makefile VPATH -magic, but this is what we have today so it's what we build with -today. -This README is probably obsolete by the time you're reading it. +## Source Tree ## -## Old Software README ## +The source code itself, for both the firmware and the software, is in +the source/ directory, with names corresponding to the canonical names +of the relevant git repositories. At the time this document was +written, a few of the repositories in question had not yet been +promoted to their expected permanent homes in the core/ or sw/ trees, +but we expect this to be temporary. Regardless, directory names +within the source/ directory tree are intended to track the canonical +names of the git repositories, whatever they may be at any given time. -Preliminary release engineering super-repository for building software -to work with the Cryptech "Alpha" board. -Primary task here is to build the PKCS #11 library and any needed -support tools for whichever platforms we support. This will involve -some packaging voodoo. +## Firmware Build Process ## -Our first targets for this are Debian and Ubuntu, probably the Jessie -and Xenial releases, respectively. If we really need to support -multiple releases for each of these platforms, the packaging mechanics -become more complicated, so we may just stop here for these platforms -and assume we can fill any odd corners using the associated source -package. +The firmware build process consists of five main phases: -Our next target for this is likely to be Mac OS X. This should be -relatively straightforward so long as we only have to support Homebrew -and we don't have to produce Homebrew "bottles" (binary packages). If -we do need to bottle, we either need one or more Mac build machines or -we need some kind of cross-compilation scheme (eg, -https://github.com/tpoechtrager/osxcross). +1. Building a "shadow" directory tree populated with symlinks back to + the source tree. This allows us to preserve binaries between + compilation runs where appropriate, without cluttering up the + source tree with various intermediate files which don't belong + there. In particular, this allows to skip the Verilog synthesis + stage when nothing there has changed, which makes the overall + build process run significantly faster. -Supporting Homebrew at all requires a bit of extra voodoo on top of -supporting Debian packaging, but none of it looks particularly -difficult, and the Debian packaging will produce the source tarball we -need for the Homebrew formula, so integrating production of these two -kinds of packaging makes some sense. +2. Synthesizing the Verilog source code into a bitstream. This phase + generates a single bitstream file, suitable for loading into the + Alpha's FPGA chip. -Windoze is not currently on the radar. In theory, MinGW would suffice -as a cross compiler if and when we have to do something about it. +3. Cross-compiling C code for the Alpha's ARM processor. This + produces two images: the bootloader, and the HSM firmware itself. -This README is probably obsolete by the time you're reading it. +4. Cross compiling C code for the Alpha's AVR MCU. This produces a + single image, which runs the tamper-response controller. + +5. Packaging all of the firmware created in the above steps into a + tarball, with some meta-data describing the package, including + SHA-2 digests of all of the firmware image files and a signature + over the entire meta-data block. + +The precise formats which show up in the firmware tarball are subject +to change. At the moment, they consist of: + +* A raw bitstream (".bit" file) for the Xilinx Artix-7 FPGA. + +* ELF and raw binary image files for the Cortex M4 ARM CPU; we supply + both .elf and .bin files because some tools want one format, some + want the other. + +* A ".hex" image for the AVR MCU. + +The overall packaging for the firmware tarball is intentionally pretty +boring: tar, gzip, gpg, and JSON. The intent is that users be able to +use our firmware tarball even if the scripts we provide are unsuitable +for some reason. + +The final result of the firmware build process is the firmware +tarball, which is written into the source/ tree for inclusion by the +software packaging phase. + + +## Software Build Process ## + +The software build process also consists of several phases, but is a +bit more open-ended than the firmware build process. Phases in the +current build process: + +1. Building a source tarball and Debian source package (".dsc"). + The Debian and Ubuntu Linux distributions are our primary + development platforms, so we need to produce packages for them in + any case, and the process of producing a Debian-family source + package produces a source tarball as one of its outputs, so we get + that for free by doing this step first. + +2. Running "pbuilder" to generate clean binary packages for the + "i386" and "amd64" architectures for Debian Jessie and Ubuntu + Xenial. + +3. Loading all of the Debian and Ubuntu packages produced above into + the staging instances of a set of APT repositories. + +4. Uploading changes from the staging repositories to our public APT + repositories. + +Planned, but missing from the above, is generation of a Homebrew +formula for Mac OS X and loading that formula along with the +corresponding source tarball to a public Homebrew "tap". At least the +initial version of this will likely be a source-only formula; in +theory, it would be possible for us to cross-compile Mac OS X binaries +without needing an expensive build farm (eg, using +https://github.com/tpoechtrager/osxcross), but we haven't sorted +through either the technical or licensing issues. + +In theory, it would also be possible to produce Windows binaries using +the MinGW cross-compilation environment, but Windows is sufficiently +different from all other platforms that even minimal Windows support +would almost certainly require extensive source code changes, so we +have not put any serious thought into build issues for Windows. -- cgit v1.2.3