README.md 18.1 KB
Newer Older
1
2
Vagabond Project
================
3
4
This project is what produced the results in "The Overhead from Combating Side-Channels in Cloud Systems using VM-Scheduling"

5
6
7
8
9
10
11
12
13
---
Prerequisites
-------------
The following needs to be installed in order for Vagabond needs to be installed:
 * Java 1.8 JDK
 * Eclipse Oxygen or above (use lower at your own risk -- untested by author)
 * `opl1261.dll` or `opl1261.so` (must be on your LD_LIBRARY_PATH)
     * This is required for running the ILP solver
     * Can be found when installing CPLEX Studio 12.61
Jonathan Shahen's avatar
Jonathan Shahen committed
14
     * Mine is set to `LD_LIBRARY_PATH=C:\Program Files\IBM\ILOG\CPLEX_Studio1261\opl\bin\x64_win64` -- (in environment variables on windows 10)
15
16
 * circuit.jar (within the lib folder and included into the build within eclipse)
     * This is required for running the CNF-SAT solver (converts Circuit SAt into CNF-SAT)
17

18
**Note:** This list is not extensive, please email one of the authors if you find a prerequisite that is not on this list.
19

20
21
22
23
---
Importing and Compiling
----------------------
It is highly suggested to use eclipse to compile and run the java source code.
24

25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
In eclipse you can add the project by following the steps below:

 1. `File -> Import`
 1. `General -> Existing Projects into Workspace`, then click `Next >`
 1. `Root Directory:` navigating to the top directory in this repo (where `.project` file is)
 1. Click `Finish`
 1. You will now have the project imported into eclipse
 1. At this point you might need to install Java 1.8 JDK and fix any classpath issues that show up within eclipse

---
Running Vagabond within Eclipse
-------------------------------
Once the Vagabond project has been imported into eclipse and there are no problems showing up, we can then run a sample test.
Follow the below steps to run a simple test to ensure that Vagabond is working. 
Later in this document we explain how to run custom tests.

 1. Within eclipse's Package Explorer, navigate to `Vagabond->src->vagabond` and open `VagabondCUI.java`
 1. The file `VagabondCUI` holds a command line interface that can be easily used within eclipse
 1. Right click on `VagabondCUI.java` and in the context menu select `Run As -> Java Application` (the wording might be slightly different depending on the eclipse setup)
 1. The previous step will make eclipse build Vagabond and then run `public static void main()` within `VagabondCUI`
 1. You will see a window open (usually on the bottom of eclipse's editor window) called `Console`
     * If you do not see the `Console` window, you can show it via the menu `Window -> Show View -> Console`
 1. In the console window, Vagabond's `Help Message` is printed (this tells you all accepted commands)
 1. Below the help message is a list of `Common Commands`; feel free to try any of these options, just copy the whole line (including the "!e") and paste it into the `Console` window
 1. Below the common commands is the `Commandline Argument` input, this is where you can write your commands
 1. You can write as many commands as you want, as many new lines, the program waits until it reads "!e" to send the command to Vagabond
 1. A simple command to try is: 
Jonathan Shahen's avatar
Jonathan Shahen committed
52
53
54
55
    ```
    -loglevel verbose -settings tests/simpletestILP-Random.properties !e
    ```
 1. To see some sample output, check out the end of this README file
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

---
Settings File
-------------
The settings file is a simple text file, where each setting is stored on a new line (can be "\n" or "\r\n").
A setting consists of a setting name, followed by an equals sign ("=") and the setting value.

Here is an example settings file:
```
# I am a comment
numberOfMachines=3
numberOfMachineSlots=3
numberOfClients=4
numberOfVMsPerClient=2
fillInEmpty=true
numberOfEpochs=10
slidingWindow=5
placement=randomPlacement
reduction=nomad
migrationBudget=3
# OPTIONAL: Can Comment these the lines below 
seed=12
```

The following are settings accepted by the program:

Jonathan Shahen's avatar
Jonathan Shahen committed
82
 * **numberOfMachines**
83
84
     * **Type:** UNSIGNED INT
     * **Description:** The number of servers/machines of the network. Machines are used to run multiple virtual machines at one time.
Jonathan Shahen's avatar
Jonathan Shahen committed
85
 * **numberOfMachineSlots**
86
87
     * **Type:** UNSIGNED INT
     * **Description:** The servers/machines are assumed homogeneous, this gives each machine a number of slots, where each slot can run 1 single virtual machine (VM).
Jonathan Shahen's avatar
Jonathan Shahen committed
88
 * **numberOfClients**
89
90
     * **Type:** UNSIGNED INT
     * **Description:** The number of clients in a system. A client can own more than 1 virtual machine (VM). We need to know the clients, because we are not considering the cases where a vm is stealing information from another vm owned by the same client.
Jonathan Shahen's avatar
Jonathan Shahen committed
91
 * **numberOfVMsPerClient**
92
93
     * **Type:** UNSIGNED INT
     * **Description:** If we assume homogeneous clients, then this number represents how many virtual machines (VMs) each client owns and that is placed in the system.
Jonathan Shahen's avatar
Jonathan Shahen committed
94
 * **fillInEmpty**
95
96
97
     * **Type:** BOOLEAN
     * **Choices:** true, false
     * **Description:** If TRUE, then any empty spaces on machines will be randomly filled by creating virtual machines (VMs) for clients. This means that the system will be 100% utilized (every slot on all machines are filled with a VM). It can mean that clients no longer have the same number of VMs (this only occurs if the free space in the system is not equal to a multiple of numberOfClients).
Jonathan Shahen's avatar
Jonathan Shahen committed
98
 * **numberOfEpochs**
99
100
     * **Type:** UNSIGNED INT
     * **Description:** An epoch is a duration of time, where at the end of an epoch we try to migrate virtual machines (VMs) to lower the total information leakage. If the migrationBudget
Jonathan Shahen's avatar
Jonathan Shahen committed
101
 * **slidingWindow**
102
103
     * **Type:** UNSIGNED INT
     * **Description:** How many epochs into the past should the calculation of total information leakage take place. Nomad has a hard-coded sliding window of 5 epochs.
Jonathan Shahen's avatar
Jonathan Shahen committed
104
 * **placement**
105
106
107
108
109
110
111
112
113
114
115
116
117
     * **Type:** STRING
     * **Description:** Defines the algorithm of how the placement of client VMs are in the system
     * **Choices:**
         * `randomPlacement` -- Randomly places VM's from each client into random machine slots. See `RandomPlacement#generateRandomPlacement()` for details.
         * `randomHalfFull` -- Random placement, but with special values for machines/slots/clients/vms such that the system has a 50% usage. Uses `equalSpreadNum`. See `PlacementMapExamples#specialTest1()` for details. 
         * `readFile` -- Reads in the placement map from a CSV formatted file. An example placement map is located `tests/nomadSubOptimalRCPlacement.csv`. Also see `vagabond.placement.readPlacementMap()` for more details.
         * `equalSpread` -- Creates a 100% utilized system, and equally spreads all client VMs over the servers, such that each machine has 1 VM for each client. This placement ignores the values in numberOfMachines/numberOfMachineSlots/numberOfClients/numberOfVMsPerClient and instead uses the value in `equalSpreadNum`. See `PlacementMapExamples#equalSpreadClients()` for details.
         * `equalSpreadHalf` -- Same as `equalSpread` but instead of 100% utilized, the system is 50% utilized. There are half as many clients as there are machines/slots/vms. See `PlacementMapExamples#equalSpreadClientsHalfFull()` for details.
         * `equalSpreadHalf2` -- Same as `equalSpreadHalf` but now it is half as many vms as there are machines/slots/clients. See `PlacementMapExamples#equalSpreadClientsHalfFull2()` for details.
         * `percentFill` -- This placement ignores the values in numberOfMachines/numberOfMachineSlots/numberOfClients/numberOfVMsPerClient and instead uses the value in `equalSpreadNum`. See `PlacementMapExamples#percentFilled()` for details.
         * `nahidTest` -- Nahid produced an initial placement map where Nomad cannot get to the optimal solution (even with infinite migration budget). In fact it does not migrate any VMs when presented with this placement map. This placement ignores the values in numberOfMachines/numberOfMachineSlots/numberOfClients/numberOfVMsPerClient. See `PlacementMapExamples#nomadSubOptimalRCPlacement()` for details.
         * `nomadTest` -- In the work's of Moon et.al. (Nomad), they have a placement map that is 50% utilized, this simulates that placement for different values of `equalSpreadNum`. See `PlacementMapExamples#nomadTest()` for details.
         * `specialTest1` -- Random placement, but with special values for machines/slots/clients/vms. Uses `equalSpreadNum`. See `PlacementMapExamples#specialTest1()` for details. 
Jonathan Shahen's avatar
Jonathan Shahen committed
118
 * **reduction**
119
120
121
122
123
124
     * **Type:** STRING
     * **Description:** This indicates which of the 3 solvers to run.
     * **Choices:**
         * `ilp` -- Reduces the placement map and migration budget to an instance of Intger Linear Programming (ILP) and solves the problem using CPLEX.
         * `cnf` -- Reduces the placement map and migration budget to an instance of Conjunctive Normal Form Boolean Satisfiability and solves the problem using lingeling.
         * `nomad` -- We have altered nomad slightly to print out necessary debugging information in order to run nomad through our software. The changes are minor, the code is still considered to be identical in functionality to the code presented in the works of Moon et.al.
Jonathan Shahen's avatar
Jonathan Shahen committed
125
 * **migrationBudget**
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
     * **Type:** INT
     * **Choices:** -1 or UNSIGNED INT
     * **Description:** The migration budget is a limit on the number of migrations allowed per EPOCH. If the migration budget is -1, then no migration budget is enforced (-1 == infinite migration budget).

---

 * **name**
     * **OPTIONAL**
     * **Type:** STRING
     * **Description:** This string is saved with the results of a test in the file `logs/vagabond.stats.csv`. It is purely cosmetic and offered as a way to keep track of similar test cases. There is no limit on the string length.
 * **comment**
     * **OPTIONAL**
     * **Type:** STRING
     * **Description:** Similar to `name`, but here is a larger string where you can identify the exact test you are performing in words. There is no limit on the string length.
 * **equalSpreadNum**
     * **SEMI-OPTIONAL**; required for many of the placement algorithms
     * **Type:** INT
     * **Description:** This is a special number that has many uses within different placement algorithms. It is suggested that `vagabond.placement.PlacementMapExamples` be analyzed to understand how this number is used (in the code it is called many names, but is represented as an integer parameter).
Jonathan Shahen's avatar
Jonathan Shahen committed
144
 * **seed**
145
146
147
148
149
150
151
152
153
154
155
156
157
158
     * **OPTIONAL**; default value is to set the random number generator's seed with the value `System.currentTimeMillis()` (`java.util.Random.setSeed(long)`)
     * **Type:** LONG 
     * **Description:** This allows for deterministic and repeatable results. On computing systems the it is customary to report an average elapsed time when reporting test results. This is because general purpose computers have other tasks they need to run along side the test and this can lead to fluctuation in the elapsed time being reported.
 * **maxCCIL**
     * **OPTIONAL**; default value of false
     * **Type:** BOOLEAN
     * **Choices:** true or false
     * **Description:** Changes how information leakage is calculated
         * **true:** the solver will optimize for the Maximum Client to Client Information Leakage
         * **false:** the solver will optimize for the Total/Summation Client to Client Information Leakage
 * **placementFile**
     * **SEMI-OPTIONAL**; this is required/used when the placement algorithm setting is `placement=readFile`
     * **Type:** FILEPATH
     * **Description:** The file path to a placement map (in CSV format), of what the initial placement should be. The values of numberOfMachines/numberOfMachineSlots/numberOfClients/numberOfVMsPerClient will be derived from the CSV file.
159
160
161
162

---
FAQ
-----
Jonathan Shahen's avatar
Jonathan Shahen committed
163
164
165
166
167
168
169
170
 * How do I view the README.md?
     * `.md` indicates the file is written in MarkDown format (like HTML but quicker and less options). Google `Online Markdown Viewer`, find a good source (don't download anything), and then copy and paste the contents of README.md into the online viewer
     * I personally use VS Code to view markdown files, but it is a full programming editor that supports many different languages...maybe too much for such a simple question.
 * What if I don't want to run the code in eclipse?
     * There is an ANT build script (`build.xml`), but that method is untested and might require more tweaking than just using eclipse
 * Who wrote the code?
     * **Jonathan Shahen** created the majority of the code in this repository
     * **Mahesh Tripunitara** wrote the code in `vagabond.reduction.cnfsat.*`
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290


---
---
Sample Output
--------------
**Running the command:** `-loglevel verbose -settings tests/simpletestILP-Random.properties !e`

**Sample Output:**
```
REMOVED THE HELP AND COMMON COMMANDS TO SAVE SPACE IN README.md

Previous Command ('!p' to use the previous command): -loglevel verbose -settings tests/simpletestILP-Random.properties !e 
Enter Commandline Argument ('!e' to run): !p
Commands: [-loglevel, verbose, -settings, tests/simpletestILP-Random.properties, !e]
2018/05/28 15:57:28.815-0400 - [vagabond.VagabondInstance.setupResultOptions #1]:
[    INFO] [OPTION] Results File: vagabond.results.txt
2018/05/28 15:57:28.829-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Input Settings: VagabondSettings{placementAlgorithm=randomPlacement, reductionAlgorithm=ilp, minimizeMaxClientToClientInfoLeak=false, numberOfEpochs=10, slidingWindow=5, migrationBudget=3, relativeMIPGapTolerance=0.0, numberOfMachines=3, numberOfMachineSlots=3, numberOfClients=4, numberOfVMsPerClient=2, fillInEmptySpots=false, seed=12, settingsFile=tests/simpletestILP-Random.properties, }
2018/05/28 15:57:28.829-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Generating Placement Map Using: randomPlacement
2018/05/28 15:57:28.832-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] PlacementMap(3 Machines; 4 Clients; 8 Total VMS)
Machine 0 {0<0,0>, 7<3,1>, 6<3,0>}
Machine 1 {2<1,0>, 3<1,1>, 1<0,1>}
Machine 2 {4<2,0>, 5<2,1>}

2018/05/28 15:57:28.835-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Starting EPOCH 0 LOOP
2018/05/28 15:57:28.838-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Reducing for EPOCH 0
2018/05/28 15:57:28.960-0400 - [vagabond.reduction.ilp.ReduceToILP.reduce #1]:
[    INFO] Using the OPL Model: D:\Work\Masters_Phd\workspace (GradSchool)\Vagabond\data\ilp_sum.mod
[ILPCustomDataSource] r = 2
2018/05/28 15:57:29.120-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Loading the Solver for EPOCH 0
2018/05/28 15:57:29.120-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Running the Solver for EPOCH 0
2018/05/28 15:57:29.181-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX OBJECTIVE: 16.0
2018/05/28 15:57:29.182-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX Time: 0.156
2018/05/28 15:57:29.184-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX Status: Optimal
Number of Moves: 1
2018/05/28 15:57:29.198-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Placement Map at the END of Epoch 0
PlacementMap(3 Machines; 4 Clients; 8 Total VMS)
Machine 0 {7<3,1>, 6<3,0>}
Machine 1 {1<0,1>, 2<1,0>, 3<1,1>}
Machine 2 {0<0,0>, 4<2,0>, 5<2,1>}

2018/05/28 15:57:29.199-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Epoch 0 took 0.364 seconds to finish
2018/05/28 15:57:29.199-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Starting EPOCH 1 LOOP
2018/05/28 15:57:29.199-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Reducing for EPOCH 1
2018/05/28 15:57:29.199-0400 - [vagabond.reduction.ilp.ReduceToILP.reduce #1]:
[    INFO] Using the OPL Model: D:\Work\Masters_Phd\workspace (GradSchool)\Vagabond\data\ilp_sum.mod
[ILPCustomDataSource] r = 2
2018/05/28 15:57:29.204-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Loading the Solver for EPOCH 1
2018/05/28 15:57:29.204-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Running the Solver for EPOCH 1
2018/05/28 15:57:29.239-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX OBJECTIVE: 24.0
2018/05/28 15:57:29.240-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX Time: 0.218
2018/05/28 15:57:29.240-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX Status: Optimal
Number of Moves: 0
2018/05/28 15:57:29.242-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Placement Map at the END of Epoch 1
PlacementMap(3 Machines; 4 Clients; 8 Total VMS)
Machine 0 {7<3,1>, 6<3,0>}
Machine 1 {1<0,1>, 2<1,0>, 3<1,1>}
Machine 2 {0<0,0>, 4<2,0>, 5<2,1>}

2018/05/28 15:57:29.242-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Epoch 1 took 0.043 seconds to finish
2018/05/28 15:57:29.242-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Starting EPOCH 2 LOOP
2018/05/28 15:57:29.244-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Reducing for EPOCH 2
2018/05/28 15:57:29.244-0400 - [vagabond.reduction.ilp.ReduceToILP.reduce #1]:
[    INFO] Using the OPL Model: D:\Work\Masters_Phd\workspace (GradSchool)\Vagabond\data\ilp_sum.mod
[ILPCustomDataSource] r = 2
2018/05/28 15:57:29.249-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Loading the Solver for EPOCH 2
2018/05/28 15:57:29.250-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Running the Solver for EPOCH 2
2018/05/28 15:57:29.307-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX OBJECTIVE: 32.0
2018/05/28 15:57:29.307-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX Time: 0.281
2018/05/28 15:57:29.307-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX Status: Optimal
Number of Moves: 0
2018/05/28 15:57:29.309-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Placement Map at the END of Epoch 2
PlacementMap(3 Machines; 4 Clients; 8 Total VMS)
Machine 0 {7<3,1>, 6<3,0>}
Machine 1 {1<0,1>, 2<1,0>, 3<1,1>}
Machine 2 {0<0,0>, 4<2,0>, 5<2,1>}



REMOVED TO SAVE SPACE WITHIN README.md (10 EPOCHS RUN)




2018/05/28 15:57:29.663-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Writing Timing Results to D:\Work\Masters_Phd\workspace (GradSchool)\Vagabond\logs\vagabond.timing.csv
2018/05/28 15:57:29.711-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Writing Results to D:\Work\Masters_Phd\workspace (GradSchool)\Vagabond\logs\vagabond.results.txt
2018/05/28 15:57:29.716-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] [EOF] Vagabond Instance done running
```