llvm
/
torch-mlir
mirror of https://github.com/llvm/torch-mlir
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
torch-mlir/build_tools/torchscript_e2e_heavydep_tests
History
Sean Silva 5b6902e31c Dual license the torch-mlir project. 
This commit (with approval from all contributors) dual licenses
the torch-mlir project under both the standard LLVM license and the
standard PyTorch license. This will facilitate moving code between
torch-mlir and the two upstream projects.

The standard file comment is now:

```
// This file is licensed under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// Also available under a BSD-style license. See LICENSE.
```

See `LICENSE` in the project root for the terms of both licenses.
 		2021-10-01 10:46:08 -07:00
..
README.md Updates to Readme.md (#334) 2021-09-28 13:50:25 -07:00
basic_mt.py Dual license the torch-mlir project. 2021-10-01 10:46:08 -07:00
bert_seq_classification.py Add BertSequenceClassification model to e2e 2021-09-30 13:30:29 -04:00
generate_serialized_tests.sh Add BertSequenceClassification model to e2e 2021-09-30 13:30:29 -04:00
main.py Dual license the torch-mlir project. 2021-10-01 10:46:08 -07:00

README.md

Additional end-to-end tests with heavy dependencies (heavydep tests)

Some end-to-end tests require additional dependencies which don't make sense to include as part of the default npcomp setup. Additionally, these dependencies often don't work with the same HEAD PyTorch version that npcomp builds against at the C++ level.

We have a self-contained script that generates all the needed artifacts from a self-contained virtual environment. It can be used like so:

# Build the virtual environment in the specified directory and generate the
# serialized test artifacts in the other specified directory.
# This command is safe to re-run if you have already built the virtual
# environment and just changed the tests.
build_tools/torchscript_e2e_heavydep_tests/generate_serialized_tests.sh \
  path/to/heavydep_venv \
  path/to/heavydep_serialized_tests

# Add the --serialized-test-dir flag to point at the directory containing the
# serialized tests. All other functionality is the same as the normal invocation
# of torchscript_e2e_test.sh, but the serialized tests will be available.
tools/torchscript_e2e_test.sh --serialized-test-dir=path/to/heavydep_serialized_tests

The tests use the same (pure-Python) test framework as the normal torchscript_e2e_test.sh, but the tests are added in build_tools/torchscript_e2e_heavydep_tests instead of frontends/pytorch/e2e_testing/torchscript.

We rely critically on serialized TorchScript compatibility across PyTorch versions to transport the tests + pure-Python compatibility of the torch API, which has worked well so far.