gitian: Add OSX build descriptors
Github-Pull: #4185 Rebased-By: Wladimir J. van der Laan Rebased-From: bb5da27, 2288206, 7fe8fe6, f76db78, ebcf375, fa1ed7c, 397e9b8
This commit is contained in:
Cory Fields
committed by
Wladimir J. van der Laan
Wladimir J. van der Laan
parent
9d97e83bf6
commit
1a97b22b9c
71
doc/README_osx.txt
Normal file
71
doc/README_osx.txt
Normal file
@@ -0,0 +1,71 @@
|
||||
Deterministic OSX Dmg Notes.
|
||||
|
||||
Working OSX DMG's are created in Linux by combining a recent clang,
|
||||
the Apple's binutils (ld, ar, etc), and DMG authoring tools.
|
||||
|
||||
Apple uses clang extensively for development and has upstreamed the necessary
|
||||
functionality so that a vanilla clang can take advantage. It supports the use
|
||||
of -F, -target, -mmacosx-version-min, and --sysroot, which are all necessary
|
||||
when building for OSX. A pre-compiled version of 3.2 is used because it was not
|
||||
available in the Precise repositories at the time this work was started. In the
|
||||
future, it can be switched to use system packages instead.
|
||||
|
||||
Apple's version of binutils (called cctools) contains lots of functionality
|
||||
missing in the FSF's binutils. In addition to extra linker options for
|
||||
frameworks and sysroots, several other tools are needed as well such as
|
||||
install_name_tool, lipo, and nmedit. These do not build under linux, so they
|
||||
have been patched to do so. The work here was used as a starting point:
|
||||
https://github.com/mingwandroid/toolchain4
|
||||
|
||||
In order to build a working toolchain, the following source packages are needed
|
||||
from Apple: cctools, dyld, and ld64.
|
||||
|
||||
Beware. This part is ugly. Very very very ugly. In the future, this should be
|
||||
broken out into a new repository and cleaned up. Additionally, the binaries
|
||||
only work when built as x86 and not x86_64. This is an especially nasty
|
||||
limitation because it must be linked with the toolchain's libLTO.so, meaning
|
||||
that the entire toolchain must be x86. Gitian x86_64 should not be used until
|
||||
this has been fixed, because it would mean that several native dependencies
|
||||
(openssl, libuuid, etc) would need to be built as x86 first.
|
||||
|
||||
These tools inject timestamps by default, which produce non-deterministic
|
||||
binaries. The ZERO_AR_DATE environment variable is used to disable that.
|
||||
|
||||
This version of cctools has been patched to use the current version of clang's
|
||||
headers and and its libLTO.so rather than those from llvmgcc, as it was
|
||||
originally done in toolchain4.
|
||||
|
||||
To complicate things further, all builds must target an Apple SDK. These SDKs
|
||||
are free to download, but not redistributable.
|
||||
To obtain it, register for a developer account, then download xcode_3.2.6_and_ios_sdk_4.3.dmg:
|
||||
https://developer.apple.com/devcenter/download.action?path=/Developer_Tools/xcode_3.2.6_and_ios_sdk_4.3__final/xcode_3.2.6_and_ios_sdk_4.3.dmg
|
||||
This file is several gigabytes in size, but only a single .pkg file inside is
|
||||
needed (MacOSX10.6.pkg). From Linux, 7-zip can be used to extract this file.
|
||||
The DMG can then be discarded.
|
||||
|
||||
The gitian descriptors build 2 sets of files: Linux tools, then Apple binaries
|
||||
which are created using these tools. The build process has been designed to
|
||||
avoid including the SDK's files in Gitian's outputs. All interim tarballs are
|
||||
fully deterministic and may be freely redistributed.
|
||||
|
||||
genisoimage is used to create the initial DMG. It is not deterministic as-is,
|
||||
so it has been patched. A system genisoimage will work fine, but it will not
|
||||
be deterministic because the file-order will change between invocations.
|
||||
The patch can be seen here:
|
||||
https://raw.githubusercontent.com/theuni/osx-cross-depends/master/patches/cdrtools/genisoimage.diff
|
||||
No effort was made to fix this cleanly, so it likely leaks memory badly. But
|
||||
it's only used for a single invocation, so that's no real concern.
|
||||
|
||||
genisoimage cannot compress DMGs, so afterwards, the 'dmg' tool from the
|
||||
libdmg-hfsplus project is used to compress it. There are several bugs in this
|
||||
tool and its maintainer has seemingly abandoned the project. It has been forked
|
||||
and is available (with fixes) here: https://github.com/theuni/libdmg-hfsplus .
|
||||
|
||||
The 'dmg' tool has the ability to create DMG's from scratch as well, but this
|
||||
functionality is broken. Only the compression feature is currently used.
|
||||
Ideally, the creation could be fixed and genisoimage would no longer be necessary.
|
||||
|
||||
Background images and other features can be added to DMG files by inserting a
|
||||
.DS_Store before creation. The easiest way to create this file is to build a
|
||||
DMG without one, move it to a device running OSX, customize the layout, then
|
||||
grab the .DS_Store file for later use. That is the approach taken here.
|
||||
@@ -33,10 +33,18 @@ Release Process
|
||||
git checkout v${VERSION}
|
||||
popd
|
||||
pushd ./gitian-builder
|
||||
mkdir -p inputs; cd inputs/
|
||||
|
||||
Register and download the Apple SDK (see OSX Readme for details)
|
||||
visit https://developer.apple.com/devcenter/download.action?path=/Developer_Tools/xcode_3.2.6_and_ios_sdk_4.3__final/xcode_3.2.6_and_ios_sdk_4.3.dmg
|
||||
|
||||
Extract MacOSX10.6.pkg using 7zip
|
||||
7z e -y xcode_3.2.6_and_ios_sdk_4.3.dmg 5.hfs
|
||||
7z -y e 5.hfs "Xcode and iOS SDK/Packages/MacOSX10.6.pkg"
|
||||
rm 5.hfs
|
||||
|
||||
Fetch and build inputs: (first time, or when dependency versions change)
|
||||
|
||||
mkdir -p inputs; cd inputs/
|
||||
wget 'http://miniupnp.free.fr/files/download.php?file=miniupnpc-1.9.tar.gz' -O miniupnpc-1.9.tar.gz
|
||||
wget 'https://www.openssl.org/source/openssl-1.0.1g.tar.gz'
|
||||
wget 'http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz'
|
||||
@@ -49,6 +57,16 @@ Release Process
|
||||
wget 'https://download.qt-project.org/official_releases/qt/5.2/5.2.0/single/qt-everywhere-opensource-src-5.2.0.tar.gz'
|
||||
wget 'https://download.qt-project.org/archive/qt/4.6/qt-everywhere-opensource-src-4.6.4.tar.gz'
|
||||
wget 'https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.bz2'
|
||||
wget 'https://github.com/mingwandroid/toolchain4/archive/10cc648683617cca8bcbeae507888099b41b530c.tar.gz'
|
||||
wget 'http://www.opensource.apple.com/tarballs/cctools/cctools-809.tar.gz'
|
||||
wget 'http://www.opensource.apple.com/tarballs/dyld/dyld-195.5.tar.gz'
|
||||
wget 'http://www.opensource.apple.com/tarballs/ld64/ld64-127.2.tar.gz'
|
||||
wget 'http://cdrkit.org/releases/cdrkit-1.1.11.tar.gz'
|
||||
wget 'https://github.com/theuni/libdmg-hfsplus/archive/libdmg-hfsplus-v0.1.tar.gz'
|
||||
wget 'http://llvm.org/releases/3.2/clang+llvm-3.2-x86-linux-ubuntu-12.04.tar.gz' -O \
|
||||
clang-llvm-3.2-x86-linux-ubuntu-12.04.tar.gz
|
||||
wget 'https://raw.githubusercontent.com/theuni/osx-cross-depends/master/patches/cdrtools/genisoimage.diff' -O \
|
||||
cdrkit-deterministic.patch
|
||||
cd ..
|
||||
./bin/gbuild ../bitcoin/contrib/gitian-descriptors/boost-linux.yml
|
||||
mv build/out/boost-*.zip inputs/
|
||||
@@ -64,6 +82,12 @@ Release Process
|
||||
mv build/out/qt-*.zip inputs/
|
||||
./bin/gbuild ../bitcoin/contrib/gitian-descriptors/protobuf-win.yml
|
||||
mv build/out/protobuf-*.zip inputs/
|
||||
./bin/gbuild ../bitcoin/contrib/gitian-descriptors/gitian-osx-native.yml
|
||||
mv build/out/osx-*.tar.gz inputs/
|
||||
./bin/gbuild ../bitcoin/contrib/gitian-descriptors/gitian-osx-depends.yml
|
||||
mv build/out/osx-*.tar.gz inputs/
|
||||
./bin/gbuild ../bitcoin/contrib/gitian-descriptors/gitian-osx-qt.yml
|
||||
mv build/out/osx-*.tar.gz inputs/
|
||||
|
||||
The expected SHA256 hashes of the intermediate inputs are:
|
||||
|
||||
@@ -96,13 +120,19 @@ Release Process
|
||||
zip -r bitcoin-${VERSION}-win-gitian.zip *
|
||||
mv bitcoin-${VERSION}-win-gitian.zip ../../../
|
||||
popd
|
||||
./bin/gbuild --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-osx-bitcoin.yml
|
||||
./bin/gsign --signer $SIGNER --release ${VERSION}-osx --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-osx-bitcoin.yml
|
||||
pushd build/out
|
||||
mv Bitcoin-Qt.dmg ../../../
|
||||
popd
|
||||
popd
|
||||
|
||||
Build output expected:
|
||||
|
||||
1. linux 32-bit and 64-bit binaries + source (bitcoin-${VERSION}-linux-gitian.zip)
|
||||
2. windows 32-bit and 64-bit binaries + installer + source (bitcoin-${VERSION}-win-gitian.zip)
|
||||
3. Gitian signatures (in gitian.sigs/${VERSION}[-win]/(your gitian key)/
|
||||
3. OSX installer (Bitcoin-Qt.dmg)
|
||||
4. Gitian signatures (in gitian.sigs/${VERSION}[-win|-osx]/(your gitian key)/
|
||||
|
||||
repackage gitian builds for release as stand-alone zip/tar/installer exe
|
||||
|
||||
@@ -119,21 +149,6 @@ repackage gitian builds for release as stand-alone zip/tar/installer exe
|
||||
zip -r bitcoin-${VERSION}-win.zip bitcoin-${VERSION}-win
|
||||
rm -rf bitcoin-${VERSION}-win
|
||||
|
||||
**Perform Mac build:**
|
||||
|
||||
OSX binaries are created by Gavin Andresen on a 64-bit, OSX 10.6 machine.
|
||||
|
||||
./autogen.sh
|
||||
SDK=$(xcode-select --print-path)/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.6.sdk
|
||||
CXXFLAGS="-mmacosx-version-min=10.6 -isysroot $SDK" ./configure --enable-upnp-default
|
||||
make
|
||||
export QTDIR=/opt/local/share/qt4 # needed to find translations/qt_*.qm files
|
||||
T=$(contrib/qt_translations.py $QTDIR/translations src/qt/locale)
|
||||
export CODESIGNARGS='--keychain ...path_to_keychain --sign "Developer ID Application: BITCOIN FOUNDATION, INC., THE"'
|
||||
python2.7 contrib/macdeploy/macdeployqtplus Bitcoin-Qt.app -sign -add-qt-tr $T -dmg -fancy contrib/macdeploy/fancy.plist
|
||||
|
||||
Build output expected: Bitcoin-Qt.dmg
|
||||
|
||||
###Next steps:
|
||||
|
||||
* Code-sign Windows -setup.exe (in a Windows virtual machine using signtool)
|
||||
|
||||
Reference in New Issue
Block a user