Effect of parameter tuning on reducing the number of queries required to perform model stealing

Effect of parameter tuning on reducing the number of queries required to perform model stealing

van Veen, Floris (TU Delft Electrical Engineering, Mathematics and Computer Science)

Roos, S. (mentor)
Hong, C. (mentor)
Huang, J. (mentor)
Lan, G. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2022-06-24

Model extraction attacks are attacks which generate a substitute model of a targeted victim neural network. It is possible to perform these attacks without a preexisting dataset, but doing so requires a very high number of queries to be sent to the victim model. This is otfen in the realm of several million queries. The more difficult the dataset, the more queries required to gain an accurate substitute model. Through each state-of-the-art model extraction algorithm, one thing that is not thoroughly optimised are the hyperparameters of the models, and optimizing them has been found to have a strong impact on accuracy of the substitute model. To attempt to reduce the number of queries required, research has been done to find the effects of optimizing hyperparameters for both MNIST and fashionMNIST datasets. This is done through grid search and random search. The results show that proper hyperparameter tuning can reduce the number of queries required to perform model stealing if they are not already optimized. Examples include requiring 125000 + queries to achieve 95% accuracy for the MNIST dataset with some hyperparameter combinations to only requiring 15000.

Adversarial attacks
model stealing
Hyperparameter Optimization

http://resolver.tudelft.nl/uuid:15c5a571-6721-4b32-baff-2ddf7352b726

Student theses

bachelor thesis

© 2022 Floris van Veen