9c1e3b8154
# Tracking [Issue](https://github.com/nod-ai/SHARK-ModelDev/issues/848) [TorchToLinalg Op Support](https://github.com/nod-ai/SHARK-ModelDev/issues/347) # Description Aten_TrilinearOp is an implementation of a "trilinear einstein sum". Essentially, just an einsum across 3 tensors. There are a few inputs: ## Tensor Inputs - i1, i2, i3 - The three input tensors for the _trilinear op. ## Expands These inputs allow you to unsqueeze an input tensor at the specified dims as a pre-processing step to make the shapes compatible for the rest of the op: - expand1: List[int], expand2: List[int], expand3: List[int] ## sumdim - sumdim: List[int] - After applying element wise multiplication, the values in sumdim denote where to collapse a dimension by summing over it ## unroll_dim - unroll_dim: int - In the PyTorch implementation, this specifies a dimension where you could slice the input tensors, multiply and sum them, then concatenate the results in an output tensor. This complicates the implementation significantly, but doesn't change the result, so I opted against it. Along with that, a previously accepted path for solving this involved reusing the AtenEinsumOp, which also would also ignore this input. # Solution After trying a bunch of more complicated approaches for it, this op actually ended up being quite simple: [See _trilinear](https://dev-discuss.pytorch.org/t/defining-the-core-aten-opset/1464) `_trilinear = (i1.unsqueeze(expand1) * i2.unsqueeze(expand2) * i3.unsqueeze(expand3)).sum(sumdim)` Wish I saw this earlier, but watcha gonna do: 🙃 ## Not Reusing AtenEinsumOp Frankly, I found multiple cases where valid inputs would have numerical mismatches for EinsumOp, even when running tests against EinsumOp directly. I think it has something to do with the singleton dimensions. Will need to look into this further, but once I realized the simplified approach, it appeared to be more reliable and much simpler. Either way (credit to @zjgarvey), there are improvements to the einsum op here. When I was originally trying to use the op, intermediate tensors were being flattened properly, but then its 0th dimension was being cast from a static dim to a dynamic dim due to integers not folding correctly in the MLIR. Figured it's worth keeping these improvements for future reusers of EinsumOp. # The zero'd out dim "bug" For some reason, if you specify a dimension in all `expands`, ```i.e. [expand1=[0], expand2=[0], expand3=[0]], [expand1=[1], expand2=[1], expand3=[1]] ``` The _trilinear op would specify `0` for that dimension in the output shape, unless it was also included in `sumdim`. This goes against the implementation of torch.einsum: ``` >>> a, b, c = [torch.rand(1, 3, 3, 3) for i in range(3)] # Simulate expand at dim=0 for all input tensors >>> torch.einsum('abcd,abcd,abcd->abcd', a, b, c).shape torch.Size([1, 3, 3, 3]) ``` And is just straight up incorrect mathematically. I considered "replacing" singleton dims with zeroed out dims, but that seemed like carrying over a bug. Instead, I included a test for the case, verified that the singleton dimensions were handled the way that torch.einsum handles it, instead of torch._trilinear, and xfailed it with a note as to why. |
||
|---|---|---|
| .github | ||
| build_tools | ||
| docs | ||
| externals | ||
| include | ||
| lib | ||
| projects | ||
| python | ||
| test | ||
| tools | ||
| utils/bazel | ||
| .clang-format | ||
| .git-blame-ignore-revs | ||
| .gitignore | ||
| .gitmodules | ||
| .pre-commit-config.yaml | ||
| .yamllint.yml | ||
| CITATION.cff | ||
| CMakeLists.txt | ||
| LICENSE | ||
| README.md | ||
| build-requirements.txt | ||
| pyproject.toml | ||
| pytorch-hash.txt | ||
| pytorch-requirements.txt | ||
| requirements.txt | ||
| setup.py | ||
| test-requirements.txt | ||
| torchvision-requirements.txt | ||
| whl-requirements.txt | ||
README.md
The Torch-MLIR Project
The Torch-MLIR project aims to provide first class compiler support from the PyTorch ecosystem to the MLIR ecosystem.
This project is participating in the LLVM Incubator process: as such, it is not part of any official LLVM release. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project is not yet endorsed as a component of LLVM.
PyTorch PyTorch is an open source machine learning framework that facilitates the seamless transition from research and prototyping to production-level deployment.
MLIR The MLIR project offers a novel approach for building extensible and reusable compiler architectures, which address the issue of software fragmentation, reduce the cost of developing domain-specific compilers, improve compilation for heterogeneous hardware, and promote compatibility between existing compilers.
Torch-MLIR Several vendors have adopted MLIR as the middle layer in their systems, enabling them to map frameworks such as PyTorch, JAX, and TensorFlow into MLIR and subsequently lower them to their target hardware. We have observed half a dozen custom lowerings from PyTorch to MLIR, making it easier for hardware vendors to focus on their unique value, rather than needing to implement yet another PyTorch frontend for MLIR. The ultimate aim is to be similar to the current hardware vendors adding LLVM target support, rather than each one implementing Clang or a C++ frontend.
All the roads from PyTorch to Torch MLIR Dialect
We have few paths to lower down to the Torch MLIR Dialect.
- ONNX as the entry points.
- Fx as the entry points
Project Communication
#torch-mlirchannel on the LLVM Discord - this is the most active communication channel- Github issues here
torch-mlirsection of LLVM Discourse
Install torch-mlir snapshot
At the time of writing, we release pre-built snapshots of torch-mlir for Python 3.11 and Python 3.10.
If you have supported Python version, the following commands initialize a virtual environment.
python3.11 -m venv mlir_venv
source mlir_venv/bin/activate
Or, if you want to switch over multiple versions of Python using conda, you can create a conda environment with Python 3.11.
conda create -n torch-mlir python=3.11
conda activate torch-mlir
python -m pip install --upgrade pip
Then, we can install torch-mlir with the corresponding torch and torchvision nightlies.
pip install --pre torch-mlir torchvision \
--extra-index-url https://download.pytorch.org/whl/nightly/cpu \
-f https://github.com/llvm/torch-mlir-release/releases/expanded_assets/dev-wheels
Using torch-mlir
Torch-MLIR is primarily a project that is integrated into compilers to bridge them to PyTorch and ONNX. If contemplating a new integration, it may be helpful to refer to existing downstreams:
While most of the project is exercised via testing paths, there are some ways that an end user can directly use the APIs without further integration:
FxImporter ResNet18
# Get the latest example if you haven't checked out the code
wget https://raw.githubusercontent.com/llvm/torch-mlir/main/projects/pt1/examples/fximporter_resnet18.py
# Run ResNet18 as a standalone script.
python projects/pt1/examples/fximporter_resnet18.py
# Output
load image from https://upload.wikimedia.org/wikipedia/commons/2/26/YellowLabradorLooking_new.jpg
...
PyTorch prediction
[('Labrador retriever', 70.65674591064453), ('golden retriever', 4.988346099853516), ('Saluki, gazelle hound', 4.477451324462891)]
torch-mlir prediction
[('Labrador retriever', 70.6567153930664), ('golden retriever', 4.988325119018555), ('Saluki, gazelle hound', 4.477458477020264)]
Repository Layout
The project follows the conventions of typical MLIR-based projects:
include/torch-mlir,libstructure for C++ MLIR compiler dialects/passes.testfor holding test code.toolsfortorch-mlir-optand such.pythontop level directory for Python code
Developers
If you would like to develop and build torch-mlir from source please look at Development Notes