llvm
/
torch-mlir
mirror of https://github.com/llvm/torch-mlir
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
torch-mlir/frontends/pytorch/test/ivalue_import
History
Sean Silva 453e29ea05 Add E2E support for tests with heavy dependencies (heavydep tests). 
The tests use the same (pure-Python) test framework as the
normal torchscript_e2e_test.sh, but the tests are added in
`build_tools/torchscript_e2e_heavydep_tests` instead of
`frontends/pytorch/e2e_testing/torchscript`. Any needed dependencies can
easily be configured in generate_serialized_tests.sh.

We add an initial machine translation model with a complex set of
dependencies to seed the curriculum there. I verified that this model
gets to the point of MLIR import (it fails there with a segfault due to
not being able to import the "Any" type).

This required moving a few files from the `torch_mlir` Python module
into multiple modules to isolate the code that depends on our C++
extensions (which now live in `torch_mlir` and
`torch_mlir_torchscript_e2e_test_configs`) from the pure Python code
(which now lives in `torch_mlir_torchscript`). This is an entirely
mechanical change, and lots of imports needed to be updated.

The dependency graph is:
```
       torch_mlir_torchscript_e2e_test_configs
                  /              |
                 /               |
                /                |
               V                 V
torch_mlir_torchscript       torch_mlir
```

The `torch_mlir_torchscript_e2e_test_configs` are then dependency-injected
into the `torch_mlir_torchscript` modules to successfully assemble a
working test harness (the code was already structured this way, but this
new file organization allows the isolation from C++ code to actually
happen).  This isolation is critical to allowing the serialized programs
to be transported across PyTorch versions and for the test harness to be
used seamlessly to generate the heavydep tests.

Also:
- Extend `_Tracer` class to support nested property (submodule) accesses.

Recommended review order:
- "user-level" docs in README.md
- code in `build_tools/torchscript_e2e_heavydep_tests`.
- changes in `torch_mlir_torchscript/e2e_test/framework.py`
- misc mechanical changes.
 		2021-08-03 14:09:56 -07:00
..
annotations Add E2E support for tests with heavy dependencies (heavydep tests). 2021-08-03 14:09:56 -07:00
README.md Rename tests to match the code they test 2021-02-25 13:31:33 -08:00
functions-that-call-methods.py Introduce native `!torch.none` type. 2021-06-14 13:30:58 -07:00
functions.py Introduce `!torch.tensor` / `!torch.vtensor` types. 2021-06-10 10:56:48 -07:00
list.py Add `!torch.int` type. 2021-06-17 07:28:23 -07:00
methods-derefine.py Add `!torch.int` type. 2021-06-17 07:28:23 -07:00
methods-locations.py Rename tests to match the code they test 2021-02-25 13:31:33 -08:00
methods.py Introduce `!torch.tensor` / `!torch.vtensor` types. 2021-06-10 10:56:48 -07:00
object-identity-error-submodule.py Rename tests to match the code they test 2021-02-25 13:31:33 -08:00
object-identity-error.py Rename tests to match the code they test 2021-02-25 13:31:33 -08:00
object-identity-torch-bug.py Add TorchList type and prim::ListConstruct #218 2021-06-10 14:31:35 -07:00
object-identity.py Add `torch.vtensor.literal` op. 2021-06-17 14:37:04 -07:00
prim.py Introduce native `!torch.none` type. 2021-06-14 13:30:58 -07:00
primitives.py Add a !torch.float type. 2021-06-17 09:24:18 -07:00
quantization.py Add `torch.vtensor.literal` op. 2021-06-17 14:37:04 -07:00
strings.py Add !torch.str type. 2021-06-15 10:10:59 -07:00
submodules-select.py Add `torch.prim.If` 2021-06-16 14:04:31 -07:00
submodules.py Add `!torch.int` type. 2021-06-17 07:28:23 -07:00
tensors.py Add `torch.vtensor.literal` op. 2021-06-17 14:37:04 -07:00
tuple.py Add `!torch.int` type. 2021-06-17 07:28:23 -07:00

README.md

ivalue_import

Most of the tests in this directory test importing of TorchScript torch::jit::Module's.

Modules are just one of many types of c10::IValue's and recursively contain c10::IValue's. Thus, the work of importing TorchScript modules is mainly about importing the wide variety of possible c10::IValue's, hence the name of this directory and the corresponding code in ivalue_importer.cpp that it exercises.