mirror of https://github.com/llvm/torch-mlir
453e29ea05
The tests use the same (pure-Python) test framework as the
normal torchscript_e2e_test.sh, but the tests are added in
`build_tools/torchscript_e2e_heavydep_tests` instead of
`frontends/pytorch/e2e_testing/torchscript`. Any needed dependencies can
easily be configured in generate_serialized_tests.sh.
We add an initial machine translation model with a complex set of
dependencies to seed the curriculum there. I verified that this model
gets to the point of MLIR import (it fails there with a segfault due to
not being able to import the "Any" type).
This required moving a few files from the `torch_mlir` Python module
into multiple modules to isolate the code that depends on our C++
extensions (which now live in `torch_mlir` and
`torch_mlir_torchscript_e2e_test_configs`) from the pure Python code
(which now lives in `torch_mlir_torchscript`). This is an entirely
mechanical change, and lots of imports needed to be updated.
The dependency graph is:
```
torch_mlir_torchscript_e2e_test_configs
/ |
/ |
/ |
V V
torch_mlir_torchscript torch_mlir
```
The `torch_mlir_torchscript_e2e_test_configs` are then dependency-injected
into the `torch_mlir_torchscript` modules to successfully assemble a
working test harness (the code was already structured this way, but this
new file organization allows the isolation from C++ code to actually
happen). This isolation is critical to allowing the serialized programs
to be transported across PyTorch versions and for the test harness to be
used seamlessly to generate the heavydep tests.
Also:
- Extend `_Tracer` class to support nested property (submodule) accesses.
Recommended review order:
- "user-level" docs in README.md
- code in `build_tools/torchscript_e2e_heavydep_tests`.
- changes in `torch_mlir_torchscript/e2e_test/framework.py`
- misc mechanical changes.
|
||
|---|---|---|
| .. | ||
| pytorch | ||
| README.md | ||
| __init__.py | ||
README.md
NPComp - Frontends
NPComp maintains in-tree frontends for various popular numeric-python based frameworks. In general these are:
- Considered optional components
- Target dialects maintained at the top-level of the project
- Maintained in isolation so as to facilitate moving them out to dedicated projects at an appropriate point of the lifecycle (i.e. if NPComp is successful as a general purpose target for such frameworks, then it may make sense to contribute/build each frontend to their respective up-stream project).
Frontends try to stylistically fit into the outer project except for when it is more clear/advantageous to align them with the conventions of the source project. This is approached on a case by case basis as needed. Deviations should be documented in a local style guide for the frontend.