With the increasing proliferation of intelligent edge devices and their applications,
machine learning has contributed significantly to regression analysis.
Over the past few decades, the sub-fields of machine learning have evolved,
such as evolutionary computing, computer vision, natural language processing,
neural networks and speech recognition. Traditionally, machine learning is
performed by collecting a huge amount of data at a centralized location, often
raising privacy concerns. More recently, edge computing has been introduced
where computation moves closer to the edge devices which decreases the latency
associated with the centralized cloud. Moreover, edge servers help in computation
offloading which decreases the data transit time to the central cloud.
However, data privacy issues remain unresolved. Hence, federated learning
has emerged as a thought-provoking technology for keeping data at the source
and obtaining a collaborative predictive model. Further, federated learning follows
rigid client-server architecture where one server communicates with many
clients. A single global server is a single point of failure. In addition, clients
have to communicate with global servers and communication latency is high
in this context. Therefore, to deal with these issues, federated learning is seen
as a critical application in edge computing. Hence, to fully exploit the enormous
data generated by devices in edge computing, edge federated learning is
a promising solution. The distributed collaborative training in EFL deals with
delay and privacy issues compared to traditional model training methods. However,
the existence of straggling devices degrades model performance. Stragglers
are caused by data and system heterogeneity. The straggler effect can be alleviated
by reinforced device selection by edge servers which can solve device
heterogeneity to some extent. But, the challenge of statistical heterogeneity
remains unsolved.
We investigate heterogeneity in data from two aspects: high-dimensional
data generated at edge devices where the number of features is greater than the
number of observations and the heterogeneity caused by partial device participation.
With a large number of features, the computation overhead on the devices
increases, causing edge devices to become stragglers. Also, the submission of
partial results causes gradients to be diverged as more local training is performed
to reach local optima. In this thesis, we introduce elastic optimization
for stragglers due to data heterogeneity in edge federated learning. Specifically,
we define the problem of stragglers in edge federated learning. Then, we formu-late an optimization problem to be solved at the edge devices. We customize the
benchmark algorithm, FedAvg, to obtain a new elastic optimization algorithm
(FedEN) which is applied in the local training of edge devices. FedEN eradicates
stragglers by achieving a balance between lasso and ridge penalization,
thereby generating sparse model updates and forcing the parameters to be as
close as possible to local optima. We experiment on the MNIST and CIFAR-
10 datasets for the proposed model. Simulated experiments demonstrate that
the proposed approach improves the run time training performance by achieving
target accuracy in less communication rounds. The results confirm the improved
performance of the FedEN approach over benchmark algorithms.