From e18e5b3d2559f5f0395ffe79416cdca3abc89310 Mon Sep 17 00:00:00 2001 From: Rob Austein Date: Sun, 14 Feb 2021 16:01:15 +0000 Subject: Start restructuring for Pelican --- markdown/BuildingFromSource.md | 229 ----------------------------------------- 1 file changed, 229 deletions(-) delete mode 100644 markdown/BuildingFromSource.md (limited to 'markdown/BuildingFromSource.md') diff --git a/markdown/BuildingFromSource.md b/markdown/BuildingFromSource.md deleted file mode 100644 index 8f2a4fd..0000000 --- a/markdown/BuildingFromSource.md +++ /dev/null @@ -1,229 +0,0 @@ -[[PageOutline]] - -# Building Cryptech Software/Firmware/Bitstream From Source - -Everything you need to build our software, firmware, and FPGA -bitstreams from source yourself is publicly available, but the process -is a bit complicated. Overall, there are two methods, one of which -our developers use while writing this stuff, the other of which we use -for the automated reproducible builds which go into our binary -distributions. Both methods eventually boil down to "get the source -code then run make", but the details differ. - -## What developers do - -We check out copies of all the several dozen separate repositories and -carefully arrange them in a tree structure which matches the official -naming scheme. Yes, really. It's tedious, but we have -[export:/user/sra/build-tools/https-sync-repos.py a script to automate this]. -Be warned that this script is a kludge which relies on parsing -XML from this Wiki; this is nasty, but reasonably stable, because the -XML itself is generated by another script. - -Once you have this tree, you can hop around within it, building -whichever bits are of interest to you. So if you want to rebuild just -the HSM firmware (the C code that runs on the ARM), you would go to -`sw/stm32` and run `make` there. - -## What we do for reproducible builds - -Reproducible builds use the same tree structure (as they must for the -various Makefiles to work properly), but the entire tree is embedded -in a git "superrepository" which also contains the release engineering -goo necessary to make the whole thing work. Do `git help submodule` -for an introduction to git's submodule mechanism. - -With this model, one just checks out a copy of -[source:/releng/alpha the superrepository], -runs `make` in its top directory, and eventually -the complete package pops out the other side. - -``` -git clone https://git.cryptech.is/releng/alpha.git -cd alpha -make -``` - -That's the good news. The bad news is that this process has higher -demands on its build environment: it expects to find the a complete -tool set, including the XiLinx synthesis tools, the several different -cross compilers for the firmware, and the `pbuilder` system for -building clean room packages for Ubuntu and Debian. - -As a compromise, one can use this source tree as if it were the -development source tree described above: just use the supermodule to -pull down everything else, but then ignore the supermodule and build -individual pieces as if you'd checked out all the repositories by -hand. - -## Skip all this git mess and just download a tarball - -There's another alternative, which is simpler than any of the above: -just download the source tarball. Since the only build environments -we support at the moment are Debian Jessie and Ubuntu Xenial, which -also happen to be environments for which we build binary packages, you -can just use APT: - -``` -apt-get source cryptech-alpha -``` - -Which will give you the same tree structure, but without all the git fun. - -## Build environment - -Our software and firmware developers use the Debian and Ubuntu Linux -distributions. Our current build box for binary packages runs Debian -Jessie. - -Our Verilog developers use various environments and have been known to -use graphical tools, but synthesis of the bitstreams that go in our -binary packages is done via the XiLinx command line tools on the same -Debian Jessie machine as the software and firmware builds. - -Which tools you need will of course depend on exactly what you're -trying to do. - -Most of the tools work on either 32-bit or 64-bit machines, but if you -intend to run the full binary package build script, you'll need a -64-bit machine (or VM) because the tools won't build 64-bit binaries -on a 32-bit machine. - -Basic tool set (not all required for every purpose, but they're all -supported Debian packages so it's usually easier just to install them -all and not worry about it): - -``` - apt-get install git pbuilder ubuntu-dev-tools rsync sudo - apt-get install python-yaml python-serial python-crypto python-ecdsa - apt-get install gcc-arm-none-eabi gdb-arm-none-eabi - apt-get install gcc-avr binutils-avr avr-libc - apt-get -t jessie-backports install debootstrap distro-info-data - apt-get install reprepro ubuntu-archive-keyring -``` - -This is not an exhaustive list, because some of the other packages we -use are pulled in by these as dependencies. - -You will also need a copy of the XiLinx tools, which is tedious enough -that it's described in a separate section, below. - -Once you have all the tools installed, you'll need a copy of the -source tree, as explained in the preceeding sections. - -pbuilder requires a bit of setup (you can skip this if you're not -trying to do the full binary package build): - -``` -for code in jessie xenial; do for arch in i386 amd64; do pbuilder-dist $code $arch create; done; done -ln -s jessie_result ~/pbuilder/jessie-amd64_result -ln -s xenial_result ~/pbuilder/xenial-amd64_result -``` - -## Installing the XiLinx tools - -XiLinx tools setup is a bit involved. You can skip this section if -you don't intend to build FPGA bitstreams. - -We use the command line versions of the XiLinx tools, but installing -them requires a graphical environment, because the XiLinx installer -and license manager are GUI tools. If you're running this on a server -and don't already have a graphical environment installed, you can get -away with something fairly minimal. For example, if you have a VNC -viewer such as "Chicken of the VNC" on your laptop, you can get away -with a fairly minimal X11 toolset: - -``` -apt-get install tightvncserver xterm icewm -``` - -If you're already running X11 on your laptop and are comfortable with -extruding that to the build machine, eg, via `ssh -Y`, you can just -use that (not recommended for long-haul use, eg, if the laptop is in -Boston and the server is in Reykjavik). - -You'll need to start by using a web browser to download the -[Xilinx ​ISE Design Suite](http://www.xilinx.com/support/download/index.html/content/xilinx/en/downloadNav/design-tools.html). - -XiLinx only supports specific versions of Red Hat and Suse Linux, but -their tools do run on Debian and Ubuntu. A few caveats: - - -* Debian and Ubuntu symlink `/bin/sh` to `/bin/dash`, which can't handle - - some of the syntax used in XiLinx's shell scripts, so you'll need to - change that symlink to point to `/bin/bash`. - - -* Although the XiLinx software can be installed as user or root, by - - default it wants to install into /opt/Xilinx, so you need to install - as root if you want to do that. - - -* The XiLinx tools are disk hogs, so if you're building a VM for this, - - you'll probably want to give it at least 30-40 GB of disk space. - -Step-by-step installation: - -1. Unpack `Xilinx_ISE_DS_Lin_14.7_1015_1.tar` (or whatever version you have). -2. In an X11 environment, cd to `Xilinx_ISE_DS_Lin_14.7_1015_1`, and run `sudo ./xsetup` -3. Click through two screens of license agreements. -4. Select `ISE WebPACK`. -5. Unselect (or leave unselected) Install Cable Drivers. -6. Go! - -Well, not quite. You'll need to convince the ISE tools that you have -a valid license to use the ISE tools. Go to -http://www.xilinx.com/products/design-tools/ise-design-suite/ise-webpack.htm, -click the `Licensing Solutions` link. On the page to which that takes -you, expand the section `Obtain a license for Free or Evaluation -product`. To download the ISE Webpack, you should have created an -account, so now you can go to the Licensing Site and use that account -to create a Certificate Based License. - -You do not need to go through the HostID dance, just say Do It. You -will then receive a certificate in email (not an X.509 certificate) -which you will be able to use. Then start the ISE Webpack by issuing -the command `ise`. Go to the Help menu and Manage Licenses. Use the -resulting new License Manager window to install the `.lic` file. This -process is complex and flakey. - -Here's -[another description of installing ISE on Ubuntu](http://www.armadeus.com/wiki/index.php?title=ISE_WebPack_installation_on_Linux). - -The `ise` binary referred to above is in `/opt/Xilinx/14.7/ISE_DS/ISE/bin/lin64/ise` -(or in `.../lin/ise`, but the pbuilder setup requires a 64-bit build machine). - -When running this remotely under tightvncserver, setup looks something like this: - -``` -vncserver :0 -geometry 1280x768 -depth 16 -localhost -export DISPLAY=:0 XAUTHORITY=~/.Xauthority -icewm& -``` - -Then, either in the same shell as the above or in an xterm in the new display - -``` -cd Xilinx_ISE_DS_Lin_14.7_1015_1 -sudo ./xsetup - -cd -/opt/Xilinx/14.7/ISE_DS/ISE/bin/lin64/ise -``` - -It turns out you don't really need to run the whole `ise` tool to -get to the license manager, you can just run - -``` -/opt/Xilinx/14.7/ISE_DS/common/bin/lin64/xlcm -manage -``` - -But you do have to source the appropriate settings file first, none of -the XiLinx tools work properly without that: - -``` -. /opt/Xilinx/14.7/ISE_DS/settings64.sh -``` -- cgit v1.2.3