llvm
/
torch-mlir
mirror of https://github.com/llvm/torch-mlir
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
torch-mlir/README.md

95 lines
5.1 KiB
Markdown
Raw Normal View History

Tidy up README and examples - update diagram to use the name "Eager Mode" instead of `torch.dispatch`, which wasn't a very accurate name - rename `resnet_inference.ipynb` to `torchscript_resnet_inference.ipynb` - this is in preparation to LTC and Eager Mode versions - remove mention of TorchFX - turns out that all TorchFX modules are actually scriptable modules, so there is literally "zero code" vs using the TorchScript path - remove LazyTensorCore example, and instead point at the current in-development `torch_mlir_ltc_backend` branch. Note: there were actually some pretty useful utilities built out in the examples directory, but they now live inside the Eager Mode `python/torch_mlir/eager_mode/ir_building.py` (and need to be rolled into a proper home with the upcoming rewrite of our top-level `torch_mlir.compile` API).
2022-03-26 07:36:57 +08:00
# The Torch-MLIR Project
Update README.md
2021-09-24 02:27:03 +08:00
Wordsmith readme
2021-09-29 04:53:33 +08:00
The Torch-MLIR project aims to provide first class compiler support from the [PyTorch](https://pytorch.org) ecosystem to the MLIR ecosystem.
License and readme changes to align with inclusion in LLVM. (#1) * Updates the LICENSE to the same verbiage as used in the circt project. * Adds the incubator disclaimer to the README. * Reworks the introduction of the README to more accurately reflect the eventual scope. * There is a fair amount of further rework of the repo that needs to take place. This is just the minimal cosmetic changes now that it is part of LLVM.
2020-08-01 11:53:09 +08:00
> This project is participating in the LLVM Incubator process: as such, it is
not part of any official LLVM release. While incubation status is not
necessarily a reflection of the completeness or stability of the code, it
does indicate that the project is not yet endorsed as a component of LLVM.
Wordsmith readme
2021-09-29 04:53:33 +08:00
[PyTorch](https://pytorch.org)
Update README.md
2021-09-24 02:27:03 +08:00
An open source machine learning framework that accelerates the path from research prototyping to production deployment.
[MLIR](https://mlir.llvm.org)
The MLIR project is a novel approach to building reusable and extensible compiler infrastructure. MLIR aims to address software fragmentation, improve compilation for heterogeneous hardware, significantly reduce the cost of building domain specific compilers, and aid in connecting existing compilers together.
Update diagram and Readme.md
2021-09-24 03:29:35 +08:00
[Torch-MLIR](https://github.com/llvm/torch-mlir)
Tidy up README and examples - update diagram to use the name "Eager Mode" instead of `torch.dispatch`, which wasn't a very accurate name - rename `resnet_inference.ipynb` to `torchscript_resnet_inference.ipynb` - this is in preparation to LTC and Eager Mode versions - remove mention of TorchFX - turns out that all TorchFX modules are actually scriptable modules, so there is literally "zero code" vs using the TorchScript path - remove LazyTensorCore example, and instead point at the current in-development `torch_mlir_ltc_backend` branch. Note: there were actually some pretty useful utilities built out in the examples directory, but they now live inside the Eager Mode `python/torch_mlir/eager_mode/ir_building.py` (and need to be rolled into a proper home with the upcoming rewrite of our top-level `torch_mlir.compile` API).
2022-03-26 07:36:57 +08:00
Multiple Vendors use MLIR as the middle layer, mapping from platform frameworks like PyTorch, JAX, and TensorFlow into MLIR and then progressively lowering down to their target hardware. We have seen half a dozen custom lowerings from PyTorch to MLIR. Having canonical lowerings from the PyTorch ecosystem to the MLIR ecosystem would provide much needed relief to hardware vendors to focus on their unique value rather than implementing yet another PyTorch frontend for MLIR. The goal is to be similar to current hardware vendors adding LLVM target support instead of each one also implementing Clang / a C++ frontend.
Update README.md
2021-09-24 02:27:03 +08:00
## All the roads from PyTorch to Torch MLIR Dialect
We have few paths to lower down to the Torch MLIR Dialect.
![Torch Lowering Architectures](Torch-MLIR.png)
Tidy up README and examples - update diagram to use the name "Eager Mode" instead of `torch.dispatch`, which wasn't a very accurate name - rename `resnet_inference.ipynb` to `torchscript_resnet_inference.ipynb` - this is in preparation to LTC and Eager Mode versions - remove mention of TorchFX - turns out that all TorchFX modules are actually scriptable modules, so there is literally "zero code" vs using the TorchScript path - remove LazyTensorCore example, and instead point at the current in-development `torch_mlir_ltc_backend` branch. Note: there were actually some pretty useful utilities built out in the examples directory, but they now live inside the Eager Mode `python/torch_mlir/eager_mode/ir_building.py` (and need to be rolled into a proper home with the upcoming rewrite of our top-level `torch_mlir.compile` API).
2022-03-26 07:36:57 +08:00
- TorchScript
This is the most tested path down to Torch MLIR Dialect, and the PyTorch ecosystem is converging on using TorchScript IR as a lingua franca.
- LazyTensorCore (Based on the PyTorch [`lazy_tensor_staging` branch](https://github.com/pytorch/pytorch/tree/lazy_tensor_staging/lazy_tensor_core))
This path provides the upcoming LTC path of capture. It is based of an unstable devel branch but is the closest way for you to adapt any existing `torch/xla` derivatives.
Update README.md
2021-09-24 02:27:03 +08:00
README: Move Project Communication section higher People might want to lurk on discord, and not even check out the code yet.
2021-11-09 03:18:00 +08:00
## Project Communication
- `#torch-mlir` channel on the LLVM [Discord](https://discord.gg/xS7Z362) - this is the most active communication channel
- Github issues [here](https://github.com/llvm/torch-mlir/issues)
- [`torch-mlir` section](https://llvm.discourse.group/c/projects-that-want-to-become-official-llvm-projects/torch-mlir/41) of LLVM Discourse
Add instructions for installing from packages.
2021-10-07 09:20:15 +08:00
Move development notes to development.md (#800) * Update README.md * Create development.md Add a separate development.md file
2022-04-27 02:28:04 +08:00
## Install torch-mlir snapshot
Update README.md (#785)
2022-04-26 22:19:48 +08:00
Remove mention of python-dev. It belongs in the development docs.
2022-05-03 00:09:40 +08:00
This installs a pre-built snapshot of torch-mlir for Python 3.7/3.8/3.9/3.10 on Linux and macOS.
Update README.md (#785)
2022-04-26 22:19:48 +08:00
```shell
python -m venv mlir_venv
source mlir_venv/bin/activate
# Some older pip installs may not be able to handle the recent PyTorch deps
python -m pip install --upgrade pip
pip install --pre torch-mlir torchvision -f https://github.com/llvm/torch-mlir/releases --extra-index-url https://download.pytorch.org/whl/nightly/cpu
# This will install the corresponding torch and torchvision nightlies
```
## Demos
### TorchScript ResNet18
Standalone script to Convert a PyTorch ResNet18 model to MLIR and run it on the CPU Backend:
```shell
# Get the latest example if you haven't checked out the code
wget https://raw.githubusercontent.com/llvm/torch-mlir/main/examples/torchscript_resnet18.py
# Run ResNet18 as a standalone script.
python examples/torchscript_resnet18.py
load image from https://upload.wikimedia.org/wikipedia/commons/2/26/YellowLabradorLooking_new.jpg
Downloading: "https://download.pytorch.org/models/resnet18-f37072fd.pth" to /home/mlir/.cache/torch/hub/checkpoints/resnet18-f37072fd.pth
100.0%
PyTorch prediction
[('Labrador retriever', 70.66319274902344), ('golden retriever', 4.956596374511719), ('Chesapeake Bay retriever', 4.195662975311279)]
torch-mlir prediction
[('Labrador retriever', 70.66320037841797), ('golden retriever', 4.956601619720459), ('Chesapeake Bay retriever', 4.195651531219482)]
```
Tidy up README and examples - update diagram to use the name "Eager Mode" instead of `torch.dispatch`, which wasn't a very accurate name - rename `resnet_inference.ipynb` to `torchscript_resnet_inference.ipynb` - this is in preparation to LTC and Eager Mode versions - remove mention of TorchFX - turns out that all TorchFX modules are actually scriptable modules, so there is literally "zero code" vs using the TorchScript path - remove LazyTensorCore example, and instead point at the current in-development `torch_mlir_ltc_backend` branch. Note: there were actually some pretty useful utilities built out in the examples directory, but they now live inside the Eager Mode `python/torch_mlir/eager_mode/ir_building.py` (and need to be rolled into a proper home with the upcoming rewrite of our top-level `torch_mlir.compile` API).
2022-03-26 07:36:57 +08:00
### LazyTensorCore
Add IREE support in TorchScript e2e tests. - Add support for "expected failures" in test reporting. The new error reports look like [this](https://gist.github.com/silvasean/6ffd95e1d55302b699673da201da210d). - We will now be able to put these tests into CI, since the harness understand which tests are expected to pass and fail. - Refactor RefBackendTestConfig to NpcompBackendTestConfig which supports both RefBackend and IREE. - Add instructions for installing IREE dependencies (both from packages and for local builds of IREE) - Add `tools/torchscript_e2e_test.sh` for invoking the e2e test harness (this makes invoking a bit easier, as it doesn't rely on a loose Python invocation).
2021-07-01 05:13:21 +08:00
Tidy up README and examples - update diagram to use the name "Eager Mode" instead of `torch.dispatch`, which wasn't a very accurate name - rename `resnet_inference.ipynb` to `torchscript_resnet_inference.ipynb` - this is in preparation to LTC and Eager Mode versions - remove mention of TorchFX - turns out that all TorchFX modules are actually scriptable modules, so there is literally "zero code" vs using the TorchScript path - remove LazyTensorCore example, and instead point at the current in-development `torch_mlir_ltc_backend` branch. Note: there were actually some pretty useful utilities built out in the examples directory, but they now live inside the Eager Mode `python/torch_mlir/eager_mode/ir_building.py` (and need to be rolled into a proper home with the upcoming rewrite of our top-level `torch_mlir.compile` API).
2022-03-26 07:36:57 +08:00
The LazyTensorCore integration is still in progress, and is being built on the
[`torch_mlir_ltc_backend` branch](https://github.com/llvm/torch-mlir/tree/torch_mlir_ltc_backend).
Add E2E support for tests with heavy dependencies (heavydep tests). The tests use the same (pure-Python) test framework as the normal torchscript_e2e_test.sh, but the tests are added in `build_tools/torchscript_e2e_heavydep_tests` instead of `frontends/pytorch/e2e_testing/torchscript`. Any needed dependencies can easily be configured in generate_serialized_tests.sh. We add an initial machine translation model with a complex set of dependencies to seed the curriculum there. I verified that this model gets to the point of MLIR import (it fails there with a segfault due to not being able to import the "Any" type). This required moving a few files from the `torch_mlir` Python module into multiple modules to isolate the code that depends on our C++ extensions (which now live in `torch_mlir` and `torch_mlir_torchscript_e2e_test_configs`) from the pure Python code (which now lives in `torch_mlir_torchscript`). This is an entirely mechanical change, and lots of imports needed to be updated. The dependency graph is: ``` torch_mlir_torchscript_e2e_test_configs / | / | / | V V torch_mlir_torchscript torch_mlir ``` The `torch_mlir_torchscript_e2e_test_configs` are then dependency-injected into the `torch_mlir_torchscript` modules to successfully assemble a working test harness (the code was already structured this way, but this new file organization allows the isolation from C++ code to actually happen). This isolation is critical to allowing the serialized programs to be transported across PyTorch versions and for the test harness to be used seamlessly to generate the heavydep tests. Also: - Extend `_Tracer` class to support nested property (submodule) accesses. Recommended review order: - "user-level" docs in README.md - code in `build_tools/torchscript_e2e_heavydep_tests`. - changes in `torch_mlir_torchscript/e2e_test/framework.py` - misc mechanical changes.
2021-07-10 03:22:45 +08:00
Eager mode description in the README and small example and ResNet18 example. (#707)
2022-03-29 12:54:06 +08:00
### Eager Mode
Eager mode with TorchMLIR is a very experimental eager mode backend for PyTorch through the torch-mlir framework.
Effectively, this mode works by compiling operator by operator as the NN is eagerly executed by PyTorch.
This mode includes a fallback to conventional PyTorch if anything in the torch-mlir compilation process fails (e.g., unsupported operator).
A simple example can be found at [eager_mode.py](examples/eager_mode.py).
A ResNet18 example can be found at [eager_mode_resnet18.py](examples/eager_mode_resnet18.py).
Updates to Readme.md (#334)
2021-09-29 04:50:25 +08:00
## Repository Layout
Add E2E support for tests with heavy dependencies (heavydep tests). The tests use the same (pure-Python) test framework as the normal torchscript_e2e_test.sh, but the tests are added in `build_tools/torchscript_e2e_heavydep_tests` instead of `frontends/pytorch/e2e_testing/torchscript`. Any needed dependencies can easily be configured in generate_serialized_tests.sh. We add an initial machine translation model with a complex set of dependencies to seed the curriculum there. I verified that this model gets to the point of MLIR import (it fails there with a segfault due to not being able to import the "Any" type). This required moving a few files from the `torch_mlir` Python module into multiple modules to isolate the code that depends on our C++ extensions (which now live in `torch_mlir` and `torch_mlir_torchscript_e2e_test_configs`) from the pure Python code (which now lives in `torch_mlir_torchscript`). This is an entirely mechanical change, and lots of imports needed to be updated. The dependency graph is: ``` torch_mlir_torchscript_e2e_test_configs / | / | / | V V torch_mlir_torchscript torch_mlir ``` The `torch_mlir_torchscript_e2e_test_configs` are then dependency-injected into the `torch_mlir_torchscript` modules to successfully assemble a working test harness (the code was already structured this way, but this new file organization allows the isolation from C++ code to actually happen). This isolation is critical to allowing the serialized programs to be transported across PyTorch versions and for the test harness to be used seamlessly to generate the heavydep tests. Also: - Extend `_Tracer` class to support nested property (submodule) accesses. Recommended review order: - "user-level" docs in README.md - code in `build_tools/torchscript_e2e_heavydep_tests`. - changes in `torch_mlir_torchscript/e2e_test/framework.py` - misc mechanical changes.
2021-07-10 03:22:45 +08:00
Updates to Readme.md (#334)
2021-09-29 04:50:25 +08:00
The project follows the conventions of typical MLIR-based projects:
Add E2E support for tests with heavy dependencies (heavydep tests). The tests use the same (pure-Python) test framework as the normal torchscript_e2e_test.sh, but the tests are added in `build_tools/torchscript_e2e_heavydep_tests` instead of `frontends/pytorch/e2e_testing/torchscript`. Any needed dependencies can easily be configured in generate_serialized_tests.sh. We add an initial machine translation model with a complex set of dependencies to seed the curriculum there. I verified that this model gets to the point of MLIR import (it fails there with a segfault due to not being able to import the "Any" type). This required moving a few files from the `torch_mlir` Python module into multiple modules to isolate the code that depends on our C++ extensions (which now live in `torch_mlir` and `torch_mlir_torchscript_e2e_test_configs`) from the pure Python code (which now lives in `torch_mlir_torchscript`). This is an entirely mechanical change, and lots of imports needed to be updated. The dependency graph is: ``` torch_mlir_torchscript_e2e_test_configs / | / | / | V V torch_mlir_torchscript torch_mlir ``` The `torch_mlir_torchscript_e2e_test_configs` are then dependency-injected into the `torch_mlir_torchscript` modules to successfully assemble a working test harness (the code was already structured this way, but this new file organization allows the isolation from C++ code to actually happen). This isolation is critical to allowing the serialized programs to be transported across PyTorch versions and for the test harness to be used seamlessly to generate the heavydep tests. Also: - Extend `_Tracer` class to support nested property (submodule) accesses. Recommended review order: - "user-level" docs in README.md - code in `build_tools/torchscript_e2e_heavydep_tests`. - changes in `torch_mlir_torchscript/e2e_test/framework.py` - misc mechanical changes.
2021-07-10 03:22:45 +08:00
Updates to Readme.md (#334)
2021-09-29 04:50:25 +08:00
* `include/torch-mlir`, `lib` structure for C++ MLIR compiler dialects/passes.
* `test` for holding test code.
* `tools` for `torch-mlir-opt` and such.
* `python` top level directory for Python code
Add E2E support for tests with heavy dependencies (heavydep tests). The tests use the same (pure-Python) test framework as the normal torchscript_e2e_test.sh, but the tests are added in `build_tools/torchscript_e2e_heavydep_tests` instead of `frontends/pytorch/e2e_testing/torchscript`. Any needed dependencies can easily be configured in generate_serialized_tests.sh. We add an initial machine translation model with a complex set of dependencies to seed the curriculum there. I verified that this model gets to the point of MLIR import (it fails there with a segfault due to not being able to import the "Any" type). This required moving a few files from the `torch_mlir` Python module into multiple modules to isolate the code that depends on our C++ extensions (which now live in `torch_mlir` and `torch_mlir_torchscript_e2e_test_configs`) from the pure Python code (which now lives in `torch_mlir_torchscript`). This is an entirely mechanical change, and lots of imports needed to be updated. The dependency graph is: ``` torch_mlir_torchscript_e2e_test_configs / | / | / | V V torch_mlir_torchscript torch_mlir ``` The `torch_mlir_torchscript_e2e_test_configs` are then dependency-injected into the `torch_mlir_torchscript` modules to successfully assemble a working test harness (the code was already structured this way, but this new file organization allows the isolation from C++ code to actually happen). This isolation is critical to allowing the serialized programs to be transported across PyTorch versions and for the test harness to be used seamlessly to generate the heavydep tests. Also: - Extend `_Tracer` class to support nested property (submodule) accesses. Recommended review order: - "user-level" docs in README.md - code in `build_tools/torchscript_e2e_heavydep_tests`. - changes in `torch_mlir_torchscript/e2e_test/framework.py` - misc mechanical changes.
2021-07-10 03:22:45 +08:00
Move development notes to development.md (#800) * Update README.md * Create development.md Add a separate development.md file
2022-04-27 02:28:04 +08:00
## Developers
If you would like to develop and build torch-mlir from source please look at [Development Notes](development.md)