1b40b6384e
As noted in the plan when this work started, we need to produce an ORT EP plugin for a downstream project, and this will necessitate a C-based ONNX importer (as opposed to the existing Python one). Because this comes with dependencies that we do not want to impart on various projects, this is optional in torch-mlir. It is also factored so that it can be used as standalone sources in downstreams that need it. Since it only depends on public C APIs on the MLIR side, this will make build coupling a lot better (since a C++ dep is not needed on the compiler and it is trivial to dynamically load). Our original plan was just to maintain this fork off to the side in our ORT plugin, but once work started, it seemed better to write it clean and contribute it upstream for anyone to use. We expect that for non-ORT use, the Python importer will have better ergonomics for most folks. I will follow-up with a test suite refactor so that we can drive the Python or C importer. This is a relatively mechanical port from Python to C, borrowing some scaffolding from the old JitIR importer. It does attempt to lay some groundwork for external data, which will need to be implemented on the Python side as well. |
||
|---|---|---|
| .github | ||
| build_tools | ||
| docs | ||
| externals | ||
| include | ||
| lib | ||
| projects | ||
| python | ||
| test | ||
| tools | ||
| utils/bazel | ||
| .clang-format | ||
| .gitignore | ||
| .gitmodules | ||
| .style.yapf | ||
| CITATION.cff | ||
| CMakeLists.txt | ||
| LICENSE | ||
| README.md | ||
| build-requirements.txt | ||
| pytorch-hash.txt | ||
| pytorch-requirements.txt | ||
| requirements.txt | ||
| setup.py | ||
| test-requirements.txt | ||
| torchvision-requirements.txt | ||
| whl-requirements.txt | ||
README.md
The Torch-MLIR Project
The Torch-MLIR project aims to provide first class compiler support from the PyTorch ecosystem to the MLIR ecosystem.
This project is participating in the LLVM Incubator process: as such, it is not part of any official LLVM release. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project is not yet endorsed as a component of LLVM.
PyTorch PyTorch is an open source machine learning framework that facilitates the seamless transition from research and prototyping to production-level deployment.
MLIR The MLIR project offers a novel approach for building extensible and reusable compiler architectures, which address the issue of software fragmentation, reduce the cost of developing domain-specific compilers, improve compilation for heterogeneous hardware, and promote compatibility between existing compilers.
Torch-MLIR Several vendors have adopted MLIR as the middle layer in their systems, enabling them to map frameworks such as PyTorch, JAX, and TensorFlow into MLIR and subsequently lower them to their target hardware. We have observed half a dozen custom lowerings from PyTorch to MLIR, making it easier for hardware vendors to focus on their unique value, rather than needing to implement yet another PyTorch frontend for MLIR. The ultimate aim is to be similar to the current hardware vendors adding LLVM target support, rather than each one implementing Clang or a C++ frontend.
All the roads from PyTorch to Torch MLIR Dialect
We have few paths to lower down to the Torch MLIR Dialect.
- TorchScript This is the most tested path down to Torch MLIR Dialect.
- LazyTensorCore Read more details here.
- We also have basic TorchDynamo/PyTorch 2.0 support, see our long-term roadmap and Thoughts on PyTorch 2.0 for more details.
Project Communication
#torch-mlirchannel on the LLVM Discord - this is the most active communication channel- Github issues here
torch-mlirsection of LLVM Discourse
Meetings
Community Meeting / Developer Hour:
- 1st and 3rd Monday of the month at 9 am PST
- 2nd and 4th Monday of the month at 5 pm PST
Office Hours:
- Every Thursday at 8:30 am PST
Meeting links can be found here.
Install torch-mlir snapshot
At the time of writing, we release pre-built snapshot of torch-mlir for Python 3.11 on Linux and macOS.
If you have Python 3.11, the following commands initialize a virtual environment.
python3.11 -m venv mlir_venv
source mlir_venv/bin/activate
Or, if you want to switch over multiple versions of Python using conda, you can create a conda environment with Python 3.11.
conda create -n torch-mlir python=3.11
conda activate torch-mlir
python -m pip install --upgrade pip
Then, we can install torch-mlir with the corresponding torch and torchvision nightlies.
pip install --pre torch-mlir torchvision \
-f https://llvm.github.io/torch-mlir/package-index/ \
--extra-index-url https://download.pytorch.org/whl/nightly/cpu
Demos
TorchScript ResNet18
Standalone script to Convert a PyTorch ResNet18 model to MLIR and run it on the CPU Backend:
# Get the latest example if you haven't checked out the code
wget https://raw.githubusercontent.com/llvm/torch-mlir/main/projects/pt1/examples/torchscript_resnet18.py
# Run ResNet18 as a standalone script.
python projects/pt1/examples/torchscript_resnet18.py
load image from https://upload.wikimedia.org/wikipedia/commons/2/26/YellowLabradorLooking_new.jpg
Downloading: "https://download.pytorch.org/models/resnet18-f37072fd.pth" to /home/mlir/.cache/torch/hub/checkpoints/resnet18-f37072fd.pth
100.0%
PyTorch prediction
[('Labrador retriever', 70.66319274902344), ('golden retriever', 4.956596374511719), ('Chesapeake Bay retriever', 4.195662975311279)]
torch-mlir prediction
[('Labrador retriever', 70.66320037841797), ('golden retriever', 4.956601619720459), ('Chesapeake Bay retriever', 4.195651531219482)]
Lazy Tensor Core
View examples here.
Repository Layout
The project follows the conventions of typical MLIR-based projects:
include/torch-mlir,libstructure for C++ MLIR compiler dialects/passes.testfor holding test code.toolsfortorch-mlir-optand such.pythontop level directory for Python code
Developers
If you would like to develop and build torch-mlir from source please look at Development Notes