README.txt 2.55 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52
WiseMove is safe reinforcement learning framework that combines hierarchical reinforcement learning and model-checking using temporal logic constraints.

Requirements
============

* Python 3.6
* Sphinx
* Please check `requirements.txt` for python package list.


Installation
============

* Run the install dependencies script: `./scripts/install_dependencies.sh` to install pip3 and required python packages.

Note: The script checks if dependencies folder exists in the project root folder. If it does, it will install from the local packages in that folder, 
else will install required packages from the internet. If you do not have an internet connection and the dependencies folder does not exist, 
you will need to run `./scripts/download_dependencies.sh` using a machine with an internet connection first and transfer that folder.

Documentation
=============

* Open `./documentation/index.html` to view the documentation
* If the file does not exist, use command `./scripts/generate_doc.sh build` to generate documentation first. Note that this requires Sphinx to be installed.

Replicate Results
=================
These are the minimum steps required to replicate the results for simple_intersection environment. For a detailed user guide, it is recommended to view the documentation.

* Run `./scripts/install_dependencies.sh` to install python dependencies.
* Low-level policies:
    * To train all low-level policies from scratch: `python3 low_level_policy_main.py --train`
    * To train a single low-level, for example wait: `python3 low_level_policy_main.py --option=wait --train`
    * To test these trained low-level policies: `python3 low_level_policy_main.py --test --saved_policy_in_root`
    * To test one of these trained low-level policies, for example wait: `python3 low_level_policy_main.py --option=wait --test --saved_policy_in_root`
* High-level policy:
    * To train high-level policy from scratch using the given low-level policies: `python3 high_level_policy_main.py --train`
    * To evaluate this trained high-level policy: `python3 high_level_policy_main.py --evaluate --saved_policy_in_root`
* To run MCTS using the high-level policy: `python3 mcts.py`

Coding Standards
================

We follow PEP8 style guidelines for coding and PEP257 for documentation.
It is not necessary to keep these in mind while coding, but before
submitting a pull request, do these two steps for each python file you 
have modified.

1. `yapf -i YOUR_MODIFIED_FILE.py`
2. `docformatter --in-place YOUR_MODIFIED_FILE.py`

`yapf` formats the code and `docformatter` formats the docstrings.