README.md 18.1 KB
Newer Older
1 2
Vagabond Project
================
3 4
This project is what produced the results in "The Overhead from Combating Side-Channels in Cloud Systems using VM-Scheduling"

5 6 7 8 9 10 11 12 13
---
Prerequisites
-------------
The following needs to be installed in order for Vagabond needs to be installed:
 * Java 1.8 JDK
 * Eclipse Oxygen or above (use lower at your own risk -- untested by author)
 * `opl1261.dll` or `opl1261.so` (must be on your LD_LIBRARY_PATH)
     * This is required for running the ILP solver
     * Can be found when installing CPLEX Studio 12.61
Jonathan Shahen's avatar
Jonathan Shahen committed
14
     * Mine is set to `LD_LIBRARY_PATH=C:\Program Files\IBM\ILOG\CPLEX_Studio1261\opl\bin\x64_win64` -- (in environment variables on windows 10)
15 16
 * circuit.jar (within the lib folder and included into the build within eclipse)
     * This is required for running the CNF-SAT solver (converts Circuit SAt into CNF-SAT)
17

18
**Note:** This list is not extensive, please email one of the authors if you find a prerequisite that is not on this list.
19

20 21 22 23
---
Importing and Compiling
----------------------
It is highly suggested to use eclipse to compile and run the java source code.
24

25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51
In eclipse you can add the project by following the steps below:

 1. `File -> Import`
 1. `General -> Existing Projects into Workspace`, then click `Next >`
 1. `Root Directory:` navigating to the top directory in this repo (where `.project` file is)
 1. Click `Finish`
 1. You will now have the project imported into eclipse
 1. At this point you might need to install Java 1.8 JDK and fix any classpath issues that show up within eclipse

---
Running Vagabond within Eclipse
-------------------------------
Once the Vagabond project has been imported into eclipse and there are no problems showing up, we can then run a sample test.
Follow the below steps to run a simple test to ensure that Vagabond is working. 
Later in this document we explain how to run custom tests.

 1. Within eclipse's Package Explorer, navigate to `Vagabond->src->vagabond` and open `VagabondCUI.java`
 1. The file `VagabondCUI` holds a command line interface that can be easily used within eclipse
 1. Right click on `VagabondCUI.java` and in the context menu select `Run As -> Java Application` (the wording might be slightly different depending on the eclipse setup)
 1. The previous step will make eclipse build Vagabond and then run `public static void main()` within `VagabondCUI`
 1. You will see a window open (usually on the bottom of eclipse's editor window) called `Console`
     * If you do not see the `Console` window, you can show it via the menu `Window -> Show View -> Console`
 1. In the console window, Vagabond's `Help Message` is printed (this tells you all accepted commands)
 1. Below the help message is a list of `Common Commands`; feel free to try any of these options, just copy the whole line (including the "!e") and paste it into the `Console` window
 1. Below the common commands is the `Commandline Argument` input, this is where you can write your commands
 1. You can write as many commands as you want, as many new lines, the program waits until it reads "!e" to send the command to Vagabond
 1. A simple command to try is: 
Jonathan Shahen's avatar
Jonathan Shahen committed
52 53 54 55
    ```
    -loglevel verbose -settings tests/simpletestILP-Random.properties !e
    ```
 1. To see some sample output, check out the end of this README file
56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81

---
Settings File
-------------
The settings file is a simple text file, where each setting is stored on a new line (can be "\n" or "\r\n").
A setting consists of a setting name, followed by an equals sign ("=") and the setting value.

Here is an example settings file:
```
# I am a comment
numberOfMachines=3
numberOfMachineSlots=3
numberOfClients=4
numberOfVMsPerClient=2
fillInEmpty=true
numberOfEpochs=10
slidingWindow=5
placement=randomPlacement
reduction=nomad
migrationBudget=3
# OPTIONAL: Can Comment these the lines below 
seed=12
```

The following are settings accepted by the program:

Jonathan Shahen's avatar
Jonathan Shahen committed
82
 * **numberOfMachines**
83 84
     * **Type:** UNSIGNED INT
     * **Description:** The number of servers/machines of the network. Machines are used to run multiple virtual machines at one time.
Jonathan Shahen's avatar
Jonathan Shahen committed
85
 * **numberOfMachineSlots**
86 87
     * **Type:** UNSIGNED INT
     * **Description:** The servers/machines are assumed homogeneous, this gives each machine a number of slots, where each slot can run 1 single virtual machine (VM).
Jonathan Shahen's avatar
Jonathan Shahen committed
88
 * **numberOfClients**
89 90
     * **Type:** UNSIGNED INT
     * **Description:** The number of clients in a system. A client can own more than 1 virtual machine (VM). We need to know the clients, because we are not considering the cases where a vm is stealing information from another vm owned by the same client.
Jonathan Shahen's avatar
Jonathan Shahen committed
91
 * **numberOfVMsPerClient**
92 93
     * **Type:** UNSIGNED INT
     * **Description:** If we assume homogeneous clients, then this number represents how many virtual machines (VMs) each client owns and that is placed in the system.
Jonathan Shahen's avatar
Jonathan Shahen committed
94
 * **fillInEmpty**
95 96 97
     * **Type:** BOOLEAN
     * **Choices:** true, false
     * **Description:** If TRUE, then any empty spaces on machines will be randomly filled by creating virtual machines (VMs) for clients. This means that the system will be 100% utilized (every slot on all machines are filled with a VM). It can mean that clients no longer have the same number of VMs (this only occurs if the free space in the system is not equal to a multiple of numberOfClients).
Jonathan Shahen's avatar
Jonathan Shahen committed
98
 * **numberOfEpochs**
99 100
     * **Type:** UNSIGNED INT
     * **Description:** An epoch is a duration of time, where at the end of an epoch we try to migrate virtual machines (VMs) to lower the total information leakage. If the migrationBudget
Jonathan Shahen's avatar
Jonathan Shahen committed
101
 * **slidingWindow**
102 103
     * **Type:** UNSIGNED INT
     * **Description:** How many epochs into the past should the calculation of total information leakage take place. Nomad has a hard-coded sliding window of 5 epochs.
Jonathan Shahen's avatar
Jonathan Shahen committed
104
 * **placement**
105 106 107 108 109 110 111 112 113 114 115 116 117
     * **Type:** STRING
     * **Description:** Defines the algorithm of how the placement of client VMs are in the system
     * **Choices:**
         * `randomPlacement` -- Randomly places VM's from each client into random machine slots. See `RandomPlacement#generateRandomPlacement()` for details.
         * `randomHalfFull` -- Random placement, but with special values for machines/slots/clients/vms such that the system has a 50% usage. Uses `equalSpreadNum`. See `PlacementMapExamples#specialTest1()` for details. 
         * `readFile` -- Reads in the placement map from a CSV formatted file. An example placement map is located `tests/nomadSubOptimalRCPlacement.csv`. Also see `vagabond.placement.readPlacementMap()` for more details.
         * `equalSpread` -- Creates a 100% utilized system, and equally spreads all client VMs over the servers, such that each machine has 1 VM for each client. This placement ignores the values in numberOfMachines/numberOfMachineSlots/numberOfClients/numberOfVMsPerClient and instead uses the value in `equalSpreadNum`. See `PlacementMapExamples#equalSpreadClients()` for details.
         * `equalSpreadHalf` -- Same as `equalSpread` but instead of 100% utilized, the system is 50% utilized. There are half as many clients as there are machines/slots/vms. See `PlacementMapExamples#equalSpreadClientsHalfFull()` for details.
         * `equalSpreadHalf2` -- Same as `equalSpreadHalf` but now it is half as many vms as there are machines/slots/clients. See `PlacementMapExamples#equalSpreadClientsHalfFull2()` for details.
         * `percentFill` -- This placement ignores the values in numberOfMachines/numberOfMachineSlots/numberOfClients/numberOfVMsPerClient and instead uses the value in `equalSpreadNum`. See `PlacementMapExamples#percentFilled()` for details.
         * `nahidTest` -- Nahid produced an initial placement map where Nomad cannot get to the optimal solution (even with infinite migration budget). In fact it does not migrate any VMs when presented with this placement map. This placement ignores the values in numberOfMachines/numberOfMachineSlots/numberOfClients/numberOfVMsPerClient. See `PlacementMapExamples#nomadSubOptimalRCPlacement()` for details.
         * `nomadTest` -- In the work's of Moon et.al. (Nomad), they have a placement map that is 50% utilized, this simulates that placement for different values of `equalSpreadNum`. See `PlacementMapExamples#nomadTest()` for details.
         * `specialTest1` -- Random placement, but with special values for machines/slots/clients/vms. Uses `equalSpreadNum`. See `PlacementMapExamples#specialTest1()` for details. 
Jonathan Shahen's avatar
Jonathan Shahen committed
118
 * **reduction**
119 120 121 122 123 124
     * **Type:** STRING
     * **Description:** This indicates which of the 3 solvers to run.
     * **Choices:**
         * `ilp` -- Reduces the placement map and migration budget to an instance of Intger Linear Programming (ILP) and solves the problem using CPLEX.
         * `cnf` -- Reduces the placement map and migration budget to an instance of Conjunctive Normal Form Boolean Satisfiability and solves the problem using lingeling.
         * `nomad` -- We have altered nomad slightly to print out necessary debugging information in order to run nomad through our software. The changes are minor, the code is still considered to be identical in functionality to the code presented in the works of Moon et.al.
Jonathan Shahen's avatar
Jonathan Shahen committed
125
 * **migrationBudget**
126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143
     * **Type:** INT
     * **Choices:** -1 or UNSIGNED INT
     * **Description:** The migration budget is a limit on the number of migrations allowed per EPOCH. If the migration budget is -1, then no migration budget is enforced (-1 == infinite migration budget).

---

 * **name**
     * **OPTIONAL**
     * **Type:** STRING
     * **Description:** This string is saved with the results of a test in the file `logs/vagabond.stats.csv`. It is purely cosmetic and offered as a way to keep track of similar test cases. There is no limit on the string length.
 * **comment**
     * **OPTIONAL**
     * **Type:** STRING
     * **Description:** Similar to `name`, but here is a larger string where you can identify the exact test you are performing in words. There is no limit on the string length.
 * **equalSpreadNum**
     * **SEMI-OPTIONAL**; required for many of the placement algorithms
     * **Type:** INT
     * **Description:** This is a special number that has many uses within different placement algorithms. It is suggested that `vagabond.placement.PlacementMapExamples` be analyzed to understand how this number is used (in the code it is called many names, but is represented as an integer parameter).
Jonathan Shahen's avatar
Jonathan Shahen committed
144
 * **seed**
145 146 147 148 149 150 151 152 153 154 155 156 157 158
     * **OPTIONAL**; default value is to set the random number generator's seed with the value `System.currentTimeMillis()` (`java.util.Random.setSeed(long)`)
     * **Type:** LONG 
     * **Description:** This allows for deterministic and repeatable results. On computing systems the it is customary to report an average elapsed time when reporting test results. This is because general purpose computers have other tasks they need to run along side the test and this can lead to fluctuation in the elapsed time being reported.
 * **maxCCIL**
     * **OPTIONAL**; default value of false
     * **Type:** BOOLEAN
     * **Choices:** true or false
     * **Description:** Changes how information leakage is calculated
         * **true:** the solver will optimize for the Maximum Client to Client Information Leakage
         * **false:** the solver will optimize for the Total/Summation Client to Client Information Leakage
 * **placementFile**
     * **SEMI-OPTIONAL**; this is required/used when the placement algorithm setting is `placement=readFile`
     * **Type:** FILEPATH
     * **Description:** The file path to a placement map (in CSV format), of what the initial placement should be. The values of numberOfMachines/numberOfMachineSlots/numberOfClients/numberOfVMsPerClient will be derived from the CSV file.
159 160 161 162

---
FAQ
-----
Jonathan Shahen's avatar
Jonathan Shahen committed
163 164 165 166 167 168 169 170
 * How do I view the README.md?
     * `.md` indicates the file is written in MarkDown format (like HTML but quicker and less options). Google `Online Markdown Viewer`, find a good source (don't download anything), and then copy and paste the contents of README.md into the online viewer
     * I personally use VS Code to view markdown files, but it is a full programming editor that supports many different languages...maybe too much for such a simple question.
 * What if I don't want to run the code in eclipse?
     * There is an ANT build script (`build.xml`), but that method is untested and might require more tweaking than just using eclipse
 * Who wrote the code?
     * **Jonathan Shahen** created the majority of the code in this repository
     * **Mahesh Tripunitara** wrote the code in `vagabond.reduction.cnfsat.*`
171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290


---
---
Sample Output
--------------
**Running the command:** `-loglevel verbose -settings tests/simpletestILP-Random.properties !e`

**Sample Output:**
```
REMOVED THE HELP AND COMMON COMMANDS TO SAVE SPACE IN README.md

Previous Command ('!p' to use the previous command): -loglevel verbose -settings tests/simpletestILP-Random.properties !e 
Enter Commandline Argument ('!e' to run): !p
Commands: [-loglevel, verbose, -settings, tests/simpletestILP-Random.properties, !e]
2018/05/28 15:57:28.815-0400 - [vagabond.VagabondInstance.setupResultOptions #1]:
[    INFO] [OPTION] Results File: vagabond.results.txt
2018/05/28 15:57:28.829-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Input Settings: VagabondSettings{placementAlgorithm=randomPlacement, reductionAlgorithm=ilp, minimizeMaxClientToClientInfoLeak=false, numberOfEpochs=10, slidingWindow=5, migrationBudget=3, relativeMIPGapTolerance=0.0, numberOfMachines=3, numberOfMachineSlots=3, numberOfClients=4, numberOfVMsPerClient=2, fillInEmptySpots=false, seed=12, settingsFile=tests/simpletestILP-Random.properties, }
2018/05/28 15:57:28.829-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Generating Placement Map Using: randomPlacement
2018/05/28 15:57:28.832-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] PlacementMap(3 Machines; 4 Clients; 8 Total VMS)
Machine 0 {0<0,0>, 7<3,1>, 6<3,0>}
Machine 1 {2<1,0>, 3<1,1>, 1<0,1>}
Machine 2 {4<2,0>, 5<2,1>}

2018/05/28 15:57:28.835-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Starting EPOCH 0 LOOP
2018/05/28 15:57:28.838-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Reducing for EPOCH 0
2018/05/28 15:57:28.960-0400 - [vagabond.reduction.ilp.ReduceToILP.reduce #1]:
[    INFO] Using the OPL Model: D:\Work\Masters_Phd\workspace (GradSchool)\Vagabond\data\ilp_sum.mod
[ILPCustomDataSource] r = 2
2018/05/28 15:57:29.120-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Loading the Solver for EPOCH 0
2018/05/28 15:57:29.120-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Running the Solver for EPOCH 0
2018/05/28 15:57:29.181-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX OBJECTIVE: 16.0
2018/05/28 15:57:29.182-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX Time: 0.156
2018/05/28 15:57:29.184-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX Status: Optimal
Number of Moves: 1
2018/05/28 15:57:29.198-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Placement Map at the END of Epoch 0
PlacementMap(3 Machines; 4 Clients; 8 Total VMS)
Machine 0 {7<3,1>, 6<3,0>}
Machine 1 {1<0,1>, 2<1,0>, 3<1,1>}
Machine 2 {0<0,0>, 4<2,0>, 5<2,1>}

2018/05/28 15:57:29.199-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Epoch 0 took 0.364 seconds to finish
2018/05/28 15:57:29.199-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Starting EPOCH 1 LOOP
2018/05/28 15:57:29.199-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Reducing for EPOCH 1
2018/05/28 15:57:29.199-0400 - [vagabond.reduction.ilp.ReduceToILP.reduce #1]:
[    INFO] Using the OPL Model: D:\Work\Masters_Phd\workspace (GradSchool)\Vagabond\data\ilp_sum.mod
[ILPCustomDataSource] r = 2
2018/05/28 15:57:29.204-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Loading the Solver for EPOCH 1
2018/05/28 15:57:29.204-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Running the Solver for EPOCH 1
2018/05/28 15:57:29.239-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX OBJECTIVE: 24.0
2018/05/28 15:57:29.240-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX Time: 0.218
2018/05/28 15:57:29.240-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX Status: Optimal
Number of Moves: 0
2018/05/28 15:57:29.242-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Placement Map at the END of Epoch 1
PlacementMap(3 Machines; 4 Clients; 8 Total VMS)
Machine 0 {7<3,1>, 6<3,0>}
Machine 1 {1<0,1>, 2<1,0>, 3<1,1>}
Machine 2 {0<0,0>, 4<2,0>, 5<2,1>}

2018/05/28 15:57:29.242-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Epoch 1 took 0.043 seconds to finish
2018/05/28 15:57:29.242-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Starting EPOCH 2 LOOP
2018/05/28 15:57:29.244-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Reducing for EPOCH 2
2018/05/28 15:57:29.244-0400 - [vagabond.reduction.ilp.ReduceToILP.reduce #1]:
[    INFO] Using the OPL Model: D:\Work\Masters_Phd\workspace (GradSchool)\Vagabond\data\ilp_sum.mod
[ILPCustomDataSource] r = 2
2018/05/28 15:57:29.249-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Loading the Solver for EPOCH 2
2018/05/28 15:57:29.250-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Running the Solver for EPOCH 2
2018/05/28 15:57:29.307-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX OBJECTIVE: 32.0
2018/05/28 15:57:29.307-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX Time: 0.281
2018/05/28 15:57:29.307-0400 - [vagabond.reduction.ilp.RunSolverILP.run #1]:
[    INFO] CPLEX Status: Optimal
Number of Moves: 0
2018/05/28 15:57:29.309-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Placement Map at the END of Epoch 2
PlacementMap(3 Machines; 4 Clients; 8 Total VMS)
Machine 0 {7<3,1>, 6<3,0>}
Machine 1 {1<0,1>, 2<1,0>, 3<1,1>}
Machine 2 {0<0,0>, 4<2,0>, 5<2,1>}



REMOVED TO SAVE SPACE WITHIN README.md (10 EPOCHS RUN)




2018/05/28 15:57:29.663-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Writing Timing Results to D:\Work\Masters_Phd\workspace (GradSchool)\Vagabond\logs\vagabond.timing.csv
2018/05/28 15:57:29.711-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] Writing Results to D:\Work\Masters_Phd\workspace (GradSchool)\Vagabond\logs\vagabond.results.txt
2018/05/28 15:57:29.716-0400 - [vagabond.VagabondInstance.run #1]:
[    INFO] [EOF] Vagabond Instance done running
```