torch-mlir/build_tools/docker/Dockerfile

55 lines
1.5 KiB
Docker
Raw Permalink Normal View History

Dockerize CI + Release builds (#1234) Gets both CI and Release builds integrated in one workflow. Mount ccache and pip cache as required for fast iterative builds Current Release docker builds still run with root perms, fix it in the future to run as the same user. There may be some corner cases left especially when switching build types etc. Docker build TEST plan: tl;dr: Build everythin: Releases (Python 3.8, 3.9, 3.10) and CIs. TM_PACKAGES="torch-mlir out-of-tree in-tree" 2.57s user 2.49s system 0% cpu 30:33.11 total Out of Tree + PyTorch binaries: Fresh build (purged cache): TM_PACKAGES="out-of-tree" 0.47s user 0.51s system 0% cpu 5:24.99 total Incremental with ccache: TM_PACKAGES="out-of-tree" 0.09s user 0.08s system 0% cpu 34.817 total Out of Tree + PyTorch from source Incremental TM_PACKAGES="out-of-tree" TM_USE_PYTORCH_BINARY=OFF 1.58s user 1.81s system 2% cpu 1:59.61 total In-Tree + PyTorch binaries: Fresh build and tests: (purge ccache) TM_PACKAGES="in-tree" 0.53s user 0.49s system 0% cpu 6:23.35 total Fresh build/ but with prior ccache TM_PACKAGES="in-tree" 0.45s user 0.66s system 0% cpu 3:57.47 total Incremental in-tree with all tests and regression tests TM_PACKAGES="in-tree" 0.16s user 0.09s system 0% cpu 2:18.52 total In-Tree + PyTorch from source Fresh build and tests: (purge ccache) TM_PACKAGES="in-tree" TM_USE_PYTORCH_BINARY=OFF 2.03s user 2.28s system 0% cpu 11:11.86 total Fresh build/ but with prior ccache TM_PACKAGES="in-tree" TM_USE_PYTORCH_BINARY=OFF 1.58s user 1.88s system 1% cpu 4:53.15 total Incremental in-tree with all tests and regression tests TM_PACKAGES="in-tree" TM_USE_PYTORCH_BINARY=OFF 1.09s user 1.10s system 1% cpu 3:29.84 total Incremental without tests TM_PACKAGES="in-tree" TM_USE_PYTORCH_BINARY=OFF TM_SKIP_TESTS=ON 1.52s user 1.42s system 3% cpu 1:15.82 total In-tree+out-of-tree + Pytorch Binaries TM_PACKAGES="out-of-tree in-tree" 0.25s user 0.18s system 0% cpu 3:01.91 total To clear all artifacts: rm -rf build build_oot llvm-build libtorch docker_venv externals/pytorch/build
2022-08-31 02:07:25 +08:00
ARG BASE_IMG=ubuntu:22.04
FROM ${BASE_IMG} as dev-base
# Disable apt-key parse waring. If someone knows how to do whatever the "proper"
# thing is then feel free. The warning complains about parsing apt-key output,
# which we're not even doing.
ARG APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE=1
ARG ARCH="x86_64"
ARG REPO_NAME="deb http://apt.llvm.org/jammy/ llvm-toolchain-jammy main"
RUN apt-get update && \
apt-get install -y \
ca-certificates \
software-properties-common \
wget \
apt-transport-https \
ccache \
curl \
cmake \
ninja-build \
git \
gnupg \
lsb-release \
python3-pip \
python3.10 \
python3.10-dev \
python3.10-venv \
unzip && \
echo $REPO_NAME >> /etc/apt/sources.list.d/llvm.list && \
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key| apt-key add - && \
apt-get update && \
apt-get install -y \
clang \
lld
######## Bazel ########
WORKDIR /install-bazel
ARG BAZEL_VERSION=5.2.0
# https://bazel.build/install/ubuntu
RUN curl -fsSL https://bazel.build/bazel-release.pub.gpg \
| gpg --dearmor >bazel-archive-keyring.gpg \
&& mv bazel-archive-keyring.gpg /usr/share/keyrings \
&& echo "deb [arch=amd64 signed-by=/usr/share/keyrings/bazel-archive-keyring.gpg] https://storage.googleapis.com/bazel-apt stable jdk1.8" \
| tee /etc/apt/sources.list.d/bazel.list \
&& apt-get update \
&& apt-get install -y "bazel=${BAZEL_VERSION?}" \
&& rm -rf /install-bazel
### Clean up
RUN apt-get clean \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /main_checkout/torch-mlir