One of the simplifications made by the pass `RefinePublicReturn`
currently only happens if the tensor in question only has one
user. However, the current method of checking this does not correctly
handle the case of a user having multiple uses of the same
tensor. This commit makes sure only unique users are considered.
This is a first step towards formalizing the set of ops in our backend
contract. The goal is to eventually formalize `torch` dialect ops into 3
categories:
1. Legal in backend contract
2. Illegal in backend contract
3. Conditionally legal in backend contract
The "conditionally legal" set are the ops that we can optionally
decompose for backends.
This patch adds relevant pass options for this throughout the compiler,
in preparation for a new set of traits which will formalize this
classification.
I recently fixed the handling of the `dim` argument in
`sum_mean_dim` (59fccab857). Therefore,
the checks that the `dim` input is `None` or `[]` are no longer needed.
This fixes a seeding issue with the [previous PR](https://github.com/llvm/torch-mlir/pull/1240) where bazel build's GHA cache is not present to begin with and one of the commands (chown) fails on it. Should get the Bazel build back to green.
This introduces a new pass LowerToBackendContract (better name very
welcome) which performs the bulk of the simplifications that we do,
such as
- shape refinement
- dtype refinement
- maximizing value semantics
- inlining global slots
- decomposing complex ops
The key difference from before is that it iterates the set of
transformations, which can help to break a number of "catch-22" issues
where one simplification depends on another, the latest example being
here:
https://github.com/llvm/torch-mlir/issues/1131
This also exposed that RefineTypes was sometimes crashing/asserting for
certain inputs. This commit hardens it a bit.
This PR adds:
- A minimal docker wrapper to the bazel GHA workflow to make it reproducible locally
- Bazel cache to speed up GHA workflows (down to ~5 minutes from ~40+minutes)
This is a no-op for non-bazel workflows and an incremental improvement.
Bumps the shape library:
- Updates the function signature for aten.arange.start_step
- upstream_shape_functions.mean_dim -> upstream_shape_functions.sum_mean_dim
* Propagate device data names
* Address PR comment
* Add example usage
* Add test for device data names
* Make TorchMlirComputation fields protected
* Add lazy backend device data name unit tests
* Disable lazy backend tests if LTC is disabled
* Add comments
When we renamed the directory containing submodules from `external` to
`externals`, we accidentally left the original name in the Github
workflow. This patch fixes the problem.
My earlier[ PR](https://github.com/llvm/torch-mlir/pull/1213) had (among other things) decoupled ubuntu and macos builds into separate matrix runs. This is not working well due to limited number of MacOS GHA VMs causing long queue times and backlog. There are two reasons causing this backlog:
1. macos arm64 builds with pytorch source are getting erratically cancelled due to resource / network constraints. This is addressed with this: https://github.com/llvm/torch-mlir/pull/1215
> "macos-arm64 (in-tree, OFF) The hosted runner: GitHub Actions 3 lost communication with the server. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error."
2. macos runs don't fail-fast when ubuntu runs fail due to being in separate matrix setups. This PR couples them again.
Building on a fresh environment + virtualenv + in-tree build errors out
becayse PyYaml isn't installed. Adding to requirements.txt fixes that.
Fixes#1173
Adding an example on how to extract MLIR output from the compilation
process in various different formats to the development documentation.
This should help developers trying to either debug torch_mlir or use it
for the purpose of extracting MLIR outputs for other testing.
Fixes#1175
* mac m1 cross compile
Add support for M1 cross compile
* Remove redundant ExecutionEngine
It is registered as part of RegisterEverything
* nuke non-universal zstd
disable LTC
The torch-mlir bazel build is [failing](https://github.com/llvm/torch-mlir/runs/7737425906?check_suite_focus=true) since [this commit](504de5e701) due to a linker failure (undefined symbol: `mlir::torch::Torch::createEraseModuleInitializerPass()`).
```
ERROR: /home/runner/.cache/bazel/_bazel_runner/db599744cd37f8c161e5034d9b9cd520/external/torch-mlir/BUILD.bazel:845:10: Linking external/torch-mlir/torch-mlir-opt failed: (Exit 1): clang failed: error executing command /usr/lib/llvm-11/bin/clang @bazel-out/k8-fastbuild/bin/external/torch-mlir/torch-mlir-opt-2.params
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
ld.lld: error: undefined symbol: mlir::torch::Torch::createEraseModuleInitializerPass()
>>> referenced by Passes.cpp
>>> bazel-out/k8-fastbuild/bin/external/torch-mlir/_objs/TorchMLIRTorchPasses/Passes.pic.o:(mlir::torch::Torch::createTorchFunctionToTorchBackendPipeline(mlir::OpPassManager&, mlir::torch::Torch::TorchLoweringPipelineOptions const&))
>>> referenced by Passes.cpp
>>> bazel-out/k8-fastbuild/bin/external/torch-mlir/_objs/TorchMLIRTorchPasses/Passes.pic.o:((anonymous namespace)::registerEraseModuleInitializerPass()::'lambda'()::operator()() const)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
This PR adds `lib/Dialect/Torch/Transforms/EraseModuleInitializer.cpp` to `TorchMLIRTorchPasses` library.
At the moment we don't gate torch-mlir PRs with bazel builds. This means bazel builds don't get run on open PRs, and so there's no good way to validate a fix PR which is meant to fix a broken bazel build. This option allows a bazel build to be manually triggered as needed on open PRs.
With llvm/llvm-project@112499f landed, `MLIR_TABLEGEN_EXE` is given as a
cache variable in the MLIR core project. Other external projects, such
as TORCH-MLIR, should not set the variable as this breaks
cross-compilation.
The Torch dialect has an include to `mlir/Dialect/Func/IR/FuncOps.h` and
should therefore have a CMake dependency to the MLIRFuncDialect.
Otherwise, the build can fail since it may occur that
`mlir/Dialect/Func/IR/FuncOps.h.inc` isn't generated yet.
Summary of changes:
- Switch to C++17 (similar to https://reviews.llvm.org/D131348)
- Update MHLO to build with LLVM commit hash 061e0189
- Replace deprecated `hasValue()` and `getValue()` with `has_value()`
and `value()` respectively (https://reviews.llvm.org/D131349)
- Use `TypedAttr` (https://reviews.llvm.org/D130092)
- Use updated assembly format of `mhlo.compare` op (commit
d03ef01e70fbf9afd0fa1976fbb7ed31838929b3 in MHLO repo)
Henry Tu
Vivek Khandelwal
Sambhav Jain
武家伟
powderluv
Sean Silva
Ramiro Leal-Cavazos
Yan Xu
Quinn Dawkins
nithinsubbiah
Jae Hoon (Antonio) Kim
Ashay Rane
Renato Golin
Prashant Kumar
Vidush Singhal
gpetters94
Ramana Radhakrishnan
Jacques Pienaar
Marius Brehler
