README.md 6.79 KB
Newer Older
Long Chung Chan's avatar
Long Chung Chan committed
1
2
# Partitioning FPGA-Optimized Systolic Arrays

LongChan's avatar
LongChan committed
3
We provide a fast optimization algorithm and a step-to-step guide on how to generate the dataset for a specific board and topologies to be used by our optimization tool.
Long Chung Chan's avatar
Long Chung Chan committed
4
5

>    *Long Chung Chan, Gurshaant Singh Malik and Nachiket Kapre*
Harry Chan's avatar
Harry Chan committed
6
7
8

>    [**"Partitioning FPGA-Optimized Systolic Arrays for Fun and Profit"**](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/optimization_algo/paper/PID6211513.pdf)

Long Chung Chan's avatar
Long Chung Chan committed
9
10
>    2019 International Conference on Field-Programmable Technology 

Harry Chan's avatar
Harry Chan committed
11
## TODO
Harry Chan's avatar
Harry Chan committed
12
13
- [x] Add a public link to the paper
- [x] Basic Demo Flow
Long Chung Chan's avatar
Long Chung Chan committed
14
15
16
17
18
19
- [] Step-by-step guide on
  - [] their own data set
  - [] running the optimzation algo on the generated dataset
- [] What each files/directories are responsible for
- [] Changes done to ScaleSim

Harry Chan's avatar
Harry Chan committed
20
## Demo
LongChan's avatar
LongChan committed
21
The following demos use pre-generated datasets and topologies that can be found in:
Long Chung Chan's avatar
Long Chung Chan committed
22

Harry Chan's avatar
Harry Chan committed
23
24
- [topologies](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/optimization_algo/topologies/) contains all the topologies descriping their respective CNN structures
- [data_source](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/optimization_algo/data_source/) contains all the cycle-accurate data generated using [SCALE sim](https://github.com/ARM-software/SCALE-Sim)
LongChan's avatar
LongChan committed
25
26
27
28
29
30
31
32
33
34
35

The instruction below will do a sweep run on each of the following networks:
   - FasterRCNN
   - Mobilenet
   - Yolo tiny
   - Googlenet
   - Alexnet
   - AlphaGoZero
   - NCF_rec
   - Resnet_50_v1

LongChan's avatar
LongChan committed
36
To obtain individual optimization result for a specific network and a specfic number of partition, please refer to the section below.
LongChan's avatar
LongChan committed
37

Harry Chan's avatar
Harry Chan committed
38
To get optimization result with:
LongChan's avatar
LongChan committed
39
1. Covariance Matrix Adaptation Evolution Strategy (CMA-es)
LongChan's avatar
LongChan committed
40
    ```bash
41
42
        cd optimization_algo/scripts
        ./sweep_nets_cma.sh
LongChan's avatar
LongChan committed
43
44
    ```

LongChan's avatar
LongChan committed
45
2. Genetic Algorithm (GA)
LongChan's avatar
LongChan committed
46
    ```bash
47
48
        cd optimization_algo/scripts
        ./sweep_nets_ga.sh
LongChan's avatar
LongChan committed
49
50
    ```

LongChan's avatar
LongChan committed
51
3. Hyperparameter Optimiztion
LongChan's avatar
LongChan committed
52
    ```bash
53
54
        cd optimization_algo/scripts
        ./sweep_nets_ho.sh
LongChan's avatar
LongChan committed
55
56
    ```

LongChan's avatar
LongChan committed
57
4. Brute Force
LongChan's avatar
LongChan committed
58
    ```bash
59
60
        cd optimization_algo/scripts
        ./sweep_nets_brute.sh
LongChan's avatar
LongChan committed
61
    ```
Long Chung Chan's avatar
Long Chung Chan committed
62

63
Result of the optimization will be added to the corresponding csv file under this [folder](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/optimization_algo/resulting_csv).
Harry Chan's avatar
Harry Chan committed
64

Harry Chan's avatar
Harry Chan committed
65
## Step-by-step detail guide
LongChan's avatar
LongChan committed
66

67
68
69
70
71
72
73
74
75
76
### 1. Custom topologies
SCALE-sim requires a `.csv` file containing the following attribut for each layer in the network:
   1. Layer name
   2. IFMAP Height
   3. IFMAP Width
   4. Filter Height
   5. Filter Width
   6. Channels
   7. Number of Filters
   8. Strides
Harry Chan's avatar
Harry Chan committed
77

78
Examples can be found under [topologies](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/optimization_algo/topologies/). If you have problem figuring out the correct topology file of a specific network, you can check out the [Netscope CNN Analyzer](https://dgschwend.github.io/netscope/quickstart.html).
Harry Chan's avatar
Harry Chan committed
79

80
81
### 2. Custom hardware model
SCALE-sim also requires a config file containing the description of your hardware model. The config file used to generate all the data in the paper is [`US_sim.cfg`](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/scaleSim/configs/US_sim.cfg). Please refer to the [SCALE sim](https://github.com/ARM-software/SCALE-Sim) for more detail on how to create your own topology file.
Long Chung Chan's avatar
Long Chung Chan committed
82

83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
### 3. Generate data source using SCALE-sim
A small modification is done on SCALE-sim to:
1. Enable multi-processing to obtain a faster generation speed on the dataset
2. Sweep through every layer with increment resource

For this reason, we have create another bash script so you don't have to worry about running SCALE-sim by yourself. 

For example, to obtain the dataset for `US_sim.cfg` with `Alexnet`:

```bash
    cd scaleSim
    ./generate_data_set.sh configs/US_sim.cfg ../topologies/960_DNN/Alexnet.csv 
```

The default value for the number of processes in parallel is `6`. This can be changed in line 257 of [scale.py](https://git.uwaterloo.ca/watcag-public/fpga-syspart/blob/master/scaleSim/scale.py). However, SCALE-sim creates temporary csv files for caching purpose, please be careful on adjusting the number to avoid `DiskOutOfSpace` error.

```python
    ...
    pool = Pool(processes = 6) # RIGHT HERE !!!
    for pro in pool.imap_unordered(self.run_mp_once, all_arr_dim_list):
        self.run_name = net_name + "_" + self.dataflow + "_" + str(pro[0]) + "x" + str(pro[1])
        self.cleanup(pro)
    pool.close()
    ...
```

After all the data are generated, all the data are spread into different files under the `outputs` directory. Run the following script to repack them into one csv file:

```bash
    cd scaleSim
    ./generate_final_csv.sh Alexnet ../optimization_algo/data_source/alexnet_mem_bound.csv 10 
```

Here are the assumption for the file name:
1. The csv file containing cycle accurate data generated from SCALE-sim: `{topology name}_mem_bound.csv`
2. The csv file containing the topology of the CNN: `{topology name}.csv`

### 4. Running script targeting specific approach
1. Covariance Matrix Adaptation Evolution Strategy (CMA-es)
    ```bash
        python3 ../approaches/cma_approach.py ${net} ${partitions} ${popsize} ${res_unit} ${strategy} ${target}
    ```
    
2. Genetic Algorithm (GA)
    ```bash
        cd optimization_algo/scripts
        ./sweep_nets_ga.sh
    ```

3. Hyperparameter Optimiztion
    ```bash
        cd optimization_algo/scripts
        ./sweep_nets_ho.sh
    ```

4. Brute Force
    ```bash
        cd optimization_algo/scripts
        ./sweep_nets_brute.sh
    ```
Long Chung Chan's avatar
Long Chung Chan committed
143

Harry Chan's avatar
Harry Chan committed
144
## Repo Breakdown
Long Chung Chan's avatar
Long Chung Chan committed
145

Harry Chan's avatar
Harry Chan committed
146
## Scale-sim change
Long Chung Chan's avatar
Long Chung Chan committed
147

LongChan's avatar
LongChan committed
148
## License
Long Chung Chan's avatar
Long Chung Chan committed
149
This tool is distributed under MIT license.
LongChan's avatar
LongChan committed
150
Copyright (c) 2019 Long Chung Chan, Gurshaant Singh Malik, Nachiket Kapre
Long Chung Chan's avatar
Long Chung Chan committed
151
152
153
154
155
156
157
158
159
160
161
162
163

<div style="text-align: justify;"> 
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
<br><br>
</div>

<div style="text-align: justify;"> 
<b>The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.</b>
<br><br>
</div>

<div style="text-align: justify;"> 
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Harry Chan's avatar
Harry Chan committed
164
</div>