Commit 0a0303bc authored by LongChan's avatar LongChan
Browse files

complete basic and deatil flow

parent 8f606025
......@@ -4,15 +4,15 @@ We provide a fast optimization algorithm and a step-to-step guide on how to gene
> *Long Chung Chan, Gurshaant Singh Malik and Nachiket Kapre*
> [**"Partitioning FPGA-Optimized Systolic Arrays for Fun and Profit"**](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/optimization_algo/paper/PID6211513.pdf)
> [**"Partitioning FPGA-Optimized Systolic Arrays for Fun and Profit"**](optimization_algo/paper/PID6211513.pdf)
> 2019 International Conference on Field-Programmable Technology
## Demo
The following demos use pre-generated datasets and topologies that can be found in:
- [topologies](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/optimization_algo/topologies/) contains all the topologies descriping their respective CNN structures
- [data_source](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/optimization_algo/data_source/) contains all the cycle-accurate data generated using [SCALE sim](https://github.com/ARM-software/SCALE-Sim)
- [topologies](optimization_algo/topologies/) contains all the topologies descriping their respective CNN structures
- [data_source](optimization_algo/data_source/) contains all the cycle-accurate data generated using [SCALE sim](https://github.com/ARM-software/SCALE-Sim)
The instruction below will do a sweep run on each of the following networks:
- FasterRCNN
......@@ -51,7 +51,7 @@ To get optimization result with:
./sweep_nets_brute.sh
```
Result of the optimization will be added to the corresponding csv file under this [folder](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/optimization_algo/resulting_csv).
Result of the optimization will be added to the corresponding csv file under this [folder](optimization_algo/resulting_csv).
## Step-by-step detail guide
......@@ -66,10 +66,10 @@ SCALE-sim requires a `.csv` file containing the following attribut for each laye
7. Number of Filters
8. Strides
Examples can be found under [topologies](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/optimization_algo/topologies/). If you have problem figuring out the correct topology file of a specific network, you can check out the [Netscope CNN Analyzer](https://dgschwend.github.io/netscope/quickstart.html).
Examples can be found under [topologies](optimization_algo/topologies/). If you have problem figuring out the correct topology file of a specific network, you can check out the [Netscope CNN Analyzer](https://dgschwend.github.io/netscope/quickstart.html).
### 2. Custom hardware model
SCALE-sim also requires a config file containing the description of your hardware model. The config file used to generate all the data in the paper is [`US_sim.cfg`](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/scaleSim/configs/US_sim.cfg). Please refer to the [SCALE sim](https://github.com/ARM-software/SCALE-Sim) for more detail on how to create your own topology file.
SCALE-sim also requires a config file containing the description of your hardware model. The config file used to generate all the data in the paper is [`US_sim.cfg`](master/scaleSim/configs/US_sim.cfg). Please refer to the [SCALE sim](https://github.com/ARM-software/SCALE-Sim) for more detail on how to create your own topology file.
### 3. Generate data source using SCALE-sim
A small modification is done on SCALE-sim to:
......@@ -85,7 +85,7 @@ For example, to obtain the dataset for `US_sim.cfg` with `Alexnet`:
./generate_data_set.sh configs/US_sim.cfg ../topologies/960_DNN/Alexnet.csv
```
The default value for the number of processes in parallel is `6`. This can be changed in line 257 of [scale.py](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/scaleSim/scale.py). However, SCALE-sim creates temporary csv files for caching purpose, please be careful on adjusting the number to avoid `DiskOutOfSpace` error.
The default value for the number of processes in parallel is `6`. This can be changed in line 257 of [scale.py](scaleSim/scale.py). However, SCALE-sim creates temporary csv files for caching purpose, please be careful on adjusting the number to avoid `DiskOutOfSpace` error.
```python
...
......@@ -111,36 +111,38 @@ Here are the assumption for the file name:
### 4. Running script targeting specific approach
Under the `optimization_algo/scripts` directory
1. Covariance Matrix Adaptation Evolution Strategy (CMA-es)
!! Please uncomment the line 349 - 364 in `cma_approach.py` to see the output
> Please uncomment the line 349 - 364 in `cma_approach.py` to see the output
```bash
# python3 ../approaches/cma_approach.py ${network name} ${number of partitions} ${population size} ${resource unit available} ${strategy} ${optimization target}
python3 ../approaches/cma_approach.py alexnet 3 100 960 allzeros DRAM_cycle
```
Result Screenshoot: ![](../screenshots/cma_result.png =200x)
Result Screenshoot: ![](screenshots/cma_result.png)
2. Genetic Algorithm (GA)
> Please uncomment the line 287 - 299 in `ga_approach.py` to see the output
```bash
# python3 ../approaches/ga_approach.py ${network name} ${number of partitions} ${elite population size} ${population size} ${resource unit available} ${optimization target}
python3 ../approaches/ga_approach.py alexnet 3 10 100 960 DRAM_cycle
```
Result Screenshoot: ![](screenshots/ga_result.png)
3. Hyperparameter Optimiztion
```bash
# python3 ../approaches/hyper_parameter_ga.py ${network name} ${number of partitions} ${resource unit available} ${target} ${max iteration}
python3 ../approaches/hyper_parameter_ga.py alexnet 3 960 DRAM_cycle 2500
```
Result Screenshoot: ![](screenshots/ho_result.png)
4. Brute Force
```bash
# python3 ../approaches/brute_force_approach.py ${network name} ${number of partitions} ${resource unit available} ${target}
python3 ../approaches/brute_force_approach.py alexnet 3 960 DRAM_cycle
```
Result Screenshoot: ![](screenshots/brute_result.png)
Result of the optimization are also added to the corresponding csv file under this [folder](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/optimization_algo/resulting_csv).
## Repo Breakdown
<!-- ## Repo Breakdown -->
## License
This tool is distributed under MIT license.
......
......@@ -283,6 +283,7 @@ class ga_approach(object):
full_latency, full_max_idx = \
self.find_max_latency([self.layers], [self.max_res_unit]*len(self.layers))
# PLEASE UNCOMMENT OUT THIS PART IF YOU NOT USING THE BASH SCRIPT WE HAVE PROVIDED
# print("================================= RESULT =================================")
# print(layer)
# print("Res mapping:")
......@@ -296,6 +297,7 @@ class ga_approach(object):
# print("==========================================================================")
# print("Throughtput Ratio:", (1/max_latency)/(1/full_latency[full_max_idx]))
# print("Latency increase:", (max_latency*self.k)/full_latency[full_max_idx])
# PLEASE UNCOMMENT OUT THIS PART IF YOU NOT USING THE BASH SCRIPT WE HAVE PROVIDED
with open(pc.RESULT_CSV_PATH+'ga.csv', 'a') as csvFile:
writer = csv.writer(csvFile, delimiter=',', lineterminator="\n")
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment