Skip to content
Snippets Groups Projects
Commit b550fb89 authored by Nishant Kumar's avatar Nishant Kumar
Browse files

Some small updates to readme.

parent 252471c8
No related branches found
No related tags found
No related merge requests found
...@@ -161,7 +161,4 @@ elif [ "$compilationTargetLower" == "porthos" ]; then ...@@ -161,7 +161,4 @@ elif [ "$compilationTargetLower" == "porthos" ]; then
# cd "$porthosFullDirPath" # cd "$porthosFullDirPath"
# make -j # make -j
echo -e "All compilation done." echo -e "All compilation done."
else
echo -e "Compilation target other than cpp and Porthos. Current compilation target = $compilationTargetHigher."
echo -e "Leaving the compiled file as ezpc file at apt location."
fi fi
- Source of Pretrained model: https://github.com/pudae/tensorflow-densenet. - Source of Pretrained model: https://github.com/pudae/tensorflow-densenet.
Download the DenseNet-121 pretrained model from [link](https://drive.google.com/open?id=0B_fUSpodN0t0eW1sVk1aeWREaDA) given in the above website and place the same in the `PreTrainedModel` folder. This should be a file of the name `tf-densenet121.tar.gz`. - Download the DenseNet-121 pretrained model from [link](https://drive.google.com/open?id=0B_fUSpodN0t0eW1sVk1aeWREaDA) given in the above website and place the same in the `PreTrainedModel` folder. This should be a file of the name `tf-densenet121.tar.gz`.
- Extract the model: `cd PreTrainedModel && tar -xvzf tf-densenet121.tar.gz && cd - ` - Extract the model: `cd PreTrainedModel && tar -xvzf tf-densenet121.tar.gz && cd - `
- Command to run the TensorFlow DenseNet model to dump the metadata: - Command to run the TensorFlow DenseNet model to dump the metadata:
`python3 DenseNet_main.py --runPrediction True --scalingFac 12 --saveImgAndWtData True` `python3 DenseNet_main.py --runPrediction True --scalingFac 12 --saveImgAndWtData True`
\ No newline at end of file
- Source of Pre-trained model: https://github.com/tensorflow/models/tree/master/official/r1/resnet -- Resnet-50 v2, fp32, Acc 76.47%, (NHWC) - Source of Pre-trained model: https://github.com/tensorflow/models/tree/master/official/r1/resnet -- Resnet-50 v2, fp32, Acc 76.47%, (NHWC).
- Download and extract the pretrained model from the above webpage and place the same in `./PreTrainedModel` folder. - Run the following to download the pretrained model from the above webpage and place the same in `./PreTrainedModel` folder:
`axel -a -n 5 -c --output ./PreTrainedModel http://download.tensorflow.org/models/official/20181001_resnet/savedmodels/resnet_v2_fp32_savedmodel_NHWC.tar.gz` `axel -a -n 5 -c --output ./PreTrainedModel http://download.tensorflow.org/models/official/20181001_resnet/savedmodels/resnet_v2_fp32_savedmodel_NHWC.tar.gz`.
- Next extract the same: - Next extract the same:
`cd PreTrainedModel && tar -xvzf resnet_v2_fp32_savedmodel_NHWC.tar.gz && cd ..` `cd PreTrainedModel && tar -xvzf resnet_v2_fp32_savedmodel_NHWC.tar.gz && cd ..`
- To run the ResNet-50 model to dump the metadata, use the following commands: - To run the ResNet-50 model to dump the metadata, use the following commands:
`python3 ResNet_main.py --runPrediction True --scalingFac 12 --saveImgAndWtData True` `python3 ResNet_main.py --runPrediction True --scalingFac 12 --saveImgAndWtData True`
\ No newline at end of file
- Source of the pretrained model: https://github.com/avoroshilov/tf-squeezenet - Source of the pretrained model: https://github.com/avoroshilov/tf-squeezenet
- Download the pretrained model: - Download the pretrained model:
`axel -a -n 5 -c --output ./PreTrainedModel https://github.com/avoroshilov/tf-squeezenet/raw/master/sqz_full.mat` `axel -a -n 5 -c --output ./PreTrainedModel https://github.com/avoroshilov/tf-squeezenet/raw/master/sqz_full.mat`
- The TensorFlow model requires scipy v1.1.0. - The TensorFlow model requires scipy v1.1.0. To install the same:
`pip3 install scipy==1.1.0` `pip3 install scipy==1.1.0`
- Run the squeezenet model to dump metadata: - Run the squeezenet model to dump metadata:
`python3 squeezenet_main.py --in ./SampleImages/n02109961_36.JPEG --saveTFMetadata True` `python3 squeezenet_main.py --in ./SampleImages/n02109961_36.JPEG --saveTFMetadata True`
- Run the squeezenet model to dump the pre-trained model and image in a format which can later be understood by Porthos. - Run the squeezenet model to dump the pre-trained model and image in a format which can later be understood by Porthos.
`python3 squeezenet_main.py --in ./SampleImages/n02109961_36.JPEG --scalingFac 12 --saveImgAndWtData True` `python3 squeezenet_main.py --in ./SampleImages/n02109961_36.JPEG --scalingFac 12 --saveImgAndWtData True`
...@@ -12,29 +12,32 @@ Below we list the packages required to get Athos up and running. We only mention ...@@ -12,29 +12,32 @@ Below we list the packages required to get Athos up and running. We only mention
The codebase is organized as follows: The codebase is organized as follows:
- `HelperScripts`: This folder contains numerous helper scripts which help from automated setup of ImageNet/CIFAR10 dataset to finding accuracy from output files. Please refer to each of the scripts for further instructions on how to use them. - `HelperScripts`: This folder contains numerous helper scripts which help from automated setup of ImageNet/CIFAR10 dataset to finding accuracy from output files. Please refer to each of the scripts for further instructions on how to use them.
- `Networks`: This folder contains the code in TensorFlow of the various benchmarks/networks we run in CrypTFlow. Among other networks, it includes code for ResNet, DenseNet, SqueezeNet for ImageNet dataset, SqueezeNet for CIFAR10 dataset, Lenet, Logistic Regression and a chest x-ray demo network. - `Networks`: This folder contains the code in TensorFlow of the various benchmarks/networks we run in CrypTFlow. Among other networks, it includes code for ResNet, DenseNet, SqueezeNet for ImageNet dataset, SqueezeNet for CIFAR10 dataset, Lenet, Logistic Regression and a chest x-ray demo network.
- `SeeDot`: This contains code for SeeDot, a high-level intermediate language on which Athos performs various optimizations before compiling to TensorFlow. - `SeeDot`: This contains code for SeeDot, a high-level intermediate language on which Athos performs various optimizations before compiling from MPC protocols.
- `TFCompiler`: This contains python modules which are called from the TensorFlow code for dumping of TensorFlow metadata (required by Athos for compilation to MPC protocols). - `TFCompiler`: This contains python modules which are called from the TensorFlow code for dumping of TensorFlow metadata (required by Athos for compilation to MPC protocols).
- `TFEzPCLibrary`: This contains library code written in EzPC for the TensorFlow nodes required during compilation. - `TFEzPCLibrary`: This contains library code written in EzPC for the TensorFlow nodes required during compilation.
- `CompileTF.sh`: The Athos compilation script. Try `./CompileTF.sh --help` for options. - `CompileTF.sh`: The Athos compilation script. Try `./CompileTF.sh --help` for options.
- `Paths.config`: This can be used to override the default folders for EzPC and Porthos. - `Paths.config`: This can be used to override the default folders for EzPC and Porthos.
# Usage # Usage
Here we provide an example on how to use Athos to compile TensorFlow based ResNet-50 code to Porthos semi-honest MPC protocol. The relevant TensorFlow code for ResNet-50 can be found in `./Networks/ResNet/ResNet_main.py`. Here we provide an example on how to use Athos to compile TensorFlow based ResNet-50 code to Porthos semi-honest 3PC protocol. The relevant TensorFlow code for ResNet-50 can be found in `./Networks/ResNet/ResNet_main.py`.
- Refer to `./Networks/ResNet/README.md` for instructions on how to download and extract the ResNet-50 pretrained model from the official TensorFlow model page. - Refer to `./Networks/ResNet/README.md` for instructions on how to download and extract the ResNet-50 pretrained model from the official TensorFlow model page.
- `cd ./Networks/ResNet && python3 ResNet_main.py --runPrediction True --scalingFac 12 --saveImgAndWtData True && cd -` - `cd ./Networks/ResNet && python3 ResNet_main.py --runPrediction True --scalingFac 12 --saveImgAndWtData True && cd -`
Runs the ResNet-50 code written in TensorFlow to dump the metadata which is required by Athos for further compilation. Runs the ResNet-50 code written in TensorFlow to dump the metadata which is required by Athos for further compilation.
This command execution should result in 2 files which will be used for further compilation - `./Networks/ResNet/graphDef.mtdata` and `./Networks/ResNet/sizeInfo.mtdata`. In addition, a file `./Networks/ResNet/ResNet_img_input.inp` should be created which contains the ResNet-50 model and image in a format which can be used later by Porthos. This command execution should result in 2 files which will be used for further compilation - `./Networks/ResNet/graphDef.mtdata` and `./Networks/ResNet/sizeInfo.mtdata`. In addition, a file `./Networks/ResNet/ResNet_img_input.inp` should be created which contains the ResNet-50 pretrained-model and image in a format which can be used later by Porthos.
- `./CompileTF.sh -b 64 -s 12 -t PORTHOS -f ./Networks/ResNet/ResNet_main.py` - `./CompileTF.sh -b 64 -s 12 -t PORTHOS -f ./Networks/ResNet/ResNet_main.py`
Using the `CompileTF.sh` script to compile the model to Porthos. This should result in creation of the file - `./Networks/ResNet/ResNet_main_64_porthos.cpp`. Using the `CompileTF.sh` script to compile the model to Porthos. This should result in creation of the file - `./Networks/ResNet/ResNet_main_64_porthos.cpp`.
- `cp ./Networks/ResNet/ResNet_main_64_porthos.cpp ../Porthos/main.cpp` - `cp ./Networks/ResNet/ResNet_main_64_porthos.cpp ../Porthos/main.cpp`
Copy the compiled file to Porthos. Copy the compiled file to Porthos.
- `cd ../Porthos && make clean && make -j` - `cd ../Porthos && make clean && make -j`
- Finally run the 3 parties. Open 3 terminals and run the following in each for the 3 parties. - Finally run the 3 parties. Open 3 terminals and run the following in each for the 3 parties.
`./party0.sh < ../Athos/Networks/ResNet/ResNet_img_input.inp` `./party0.sh < ../Athos/Networks/ResNet/ResNet_img_input.inp` ,
`./party1.sh` `./party1.sh` ,
`./party2.sh` `./party2.sh`.
Once the above runs, the final answer for prediction should appear in the output of party1. For the sample image, this answer should be 249 for ResNet and 248 for DenseNet/SqueezeNet. Once the above runs, the final answer for prediction should appear in the output of party1. For the sample image, this answer should be 249 for ResNet and 248 for DenseNet/SqueezeNet.
Instructions on how to run the particular TensorFlow model in `./Networks` can vary. Please refer to the appropriate readme in each model to get more insights. But once that is done, the further compilation commands are the same. Instructions on how to run the particular TensorFlow model in `./Networks` can vary. Please refer to the appropriate readme in each model folder to get more insights. But once that is done, the further compilation commands are the same.
...@@ -10,19 +10,18 @@ This folder contains code for Porthos - a semi-honest 3 party secure computation ...@@ -10,19 +10,18 @@ This folder contains code for Porthos - a semi-honest 3 party secure computation
# Running the protocol # Running the protocol
- To compile use `make clean && make -j`. - To compile use `make clean && make -j`.
- Currently the codebase contains precompiled code the following 3 neural networks: ResNet-50, DenseNet-121 and SqueezeNet for ImageNet, checked into the following folder: `./src/example_neural_nets`. Toggle the flag in `./src/example_neural_nets/network_config.h` to switch the network which runs. Note that if there is more than network flag uncommented or if there is already a main file in src, the compilation will error out. - Currently the codebase contains precompiled code the following 3 neural networks: ResNet-50, DenseNet-121 and SqueezeNet for ImageNet, checked into the following folder: `./src/example_neural_nets`. Toggle the flag in `./src/example_neural_nets/network_config.h` to switch the network which runs. Note that if there is more than network flag uncommented or if there is already a main file in src, the compilation will error out.
- For example, to run the ResNet-50 code, use the following commands: - To run for example the ResNet-50 code, use the following commands:
`./party0.sh < ../Athos/Networks/ResNet/ResNet_img_input.inp` `./party0.sh < ../Athos/Networks/ResNet/ResNet_img_input.inp`,
`./party1.sh` `./party1.sh`, and
`./party2.sh` `./party2.sh`.
Note that the first command makes use of input generated from Athos. Please refer to the Athos readme on how to do the same. Note that the first command makes use of input generated from Athos. Please refer to the Athos readme on how to do the same.
# Notes # Notes
- If porthosSecretType != uint64_t, porthosSecretType multiplication function won't work. - If porthosSecretType != uint64_t, porthosSecretType multiplication function won't work.
- Possible instructions for installation of libssl on Mac, though untested. - Possible instructions for installation of libssl on Mac, though untested.
* brew install libssl -- brew install libssl
* cd /usr/local/include -- cd /usr/local/include
* ln -s ../opt/openssl/include/openssl . -- ln -s ../opt/openssl/include/openssl .
Also, use the second build command for Mac to be safe. Read the Caveats part of brew install instructions, since Apple has deprecated the use of OpenSSL. Also, use the second build command for Mac to be safe. Read the Caveats part of brew install instructions, since Apple has deprecated the use of OpenSSL.
- libmiracl.a is compiled locally, if it does not work try the libmiracl_old.a or download the source files from https://github.com/miracl/MIRACL.git and compile miracl.a yourself (and rename to libmriacl.a when copying into this repo) - libmiracl.a is compiled locally, if it does not work try the libmiracl_old.a or download the source files from https://github.com/miracl/MIRACL.git and compile miracl.a yourself (and rename to libmriacl.a when copying into this repo)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment