mirror of https://github.com/llvm/torch-mlir
ci: introduce read-only and read-write PyTorch build caches (#1546)
Until recently, we had to either risk feature branches creating PyTorch build caches (which were unusable by the main branch or other parallel feature branches because of GitHub's rules around sharing caches among branches) or we had to limit the PyTorch build caches to only the main branch, causing CI runs on feature branches to be terribly slow because they had to rebuild PyTorch each time. This patch enables the best of both worlds, by using a fork (github.com/ashay/cache) of the GitHub's cache action, where the fork adds an option (called `save`) which, when set, uploads a new cache entry. We thus set this `save` flag only when we're building PyTorch from source in Torch-MLIR's main branch, whereas all other builds set this `save` flag to `false`. The ability to conditionally update the cache has been an oft-requested feature on the original (github.com/actions/cache) repository and multiple unmerged PRs exist to allow conditional cache updates, so it is likely that using the fork is only a temporary solution.pull/1549/head
parent
79871040c9
commit
031d127940
|
@ -71,7 +71,7 @@ jobs:
|
||||||
- name: Update PyTorch Build Cache (if running on main branch)
|
- name: Update PyTorch Build Cache (if running on main branch)
|
||||||
if: github.ref_name == 'main'
|
if: github.ref_name == 'main'
|
||||||
id: cache-pytorch
|
id: cache-pytorch
|
||||||
uses: actions/cache@v3
|
uses: ashay/cache@v1
|
||||||
with:
|
with:
|
||||||
path: ${{ github.workspace }}/build_tools/python_deploy/wheelhouse
|
path: ${{ github.workspace }}/build_tools/python_deploy/wheelhouse
|
||||||
key: ubuntu-x86_64-pytorch-${{ env.PT_HASH }}
|
key: ${{ runner.os }}-pytorch-${{ env.PT_HASH }}
|
||||||
|
|
|
@ -73,13 +73,14 @@ jobs:
|
||||||
with:
|
with:
|
||||||
arch: x64
|
arch: x64
|
||||||
|
|
||||||
- name: Cache PyTorch Build (if running on main branch)
|
- name: Cache PyTorch Build
|
||||||
if: github.ref_name == 'main' && matrix.os-arch != 'windows-x86_64'
|
if: matrix.os-arch != 'windows-x86_64'
|
||||||
id: cache-pytorch
|
id: cache-pytorch
|
||||||
uses: actions/cache@v3
|
uses: ashay/cache@v1
|
||||||
with:
|
with:
|
||||||
path: ${{ github.workspace }}/build_tools/python_deploy/wheelhouse
|
path: ${{ github.workspace }}/build_tools/python_deploy/wheelhouse
|
||||||
key: ${{ matrix.os-arch }}-pytorch-${{ env.PT_HASH }}
|
key: ${{ runner.os }}-pytorch-${{ env.PT_HASH }}
|
||||||
|
save: ${{ github.ref_name == 'main' && matrix.torch-binary == 'OFF' }}
|
||||||
|
|
||||||
- name: Build and Test os-arch='ubuntu-x86_64' llvm-build='${{ matrix.llvm-build }}' torch-binary='${{ matrix.torch-binary }}'
|
- name: Build and Test os-arch='ubuntu-x86_64' llvm-build='${{ matrix.llvm-build }}' torch-binary='${{ matrix.torch-binary }}'
|
||||||
if: ${{ matrix.os-arch == 'ubuntu-x86_64' }}
|
if: ${{ matrix.os-arch == 'ubuntu-x86_64' }}
|
||||||
|
|
Loading…
Reference in New Issue