2023
Open Access-Veröffentlichung
Open Access Green
Bachelorarbeit
Published
Zusammenfassung

In this thesis, we first lay some theoretical groundwork before motivating and discussing the stochastic gradient descent method along with its variations. We then analyze some popular step size strategies with a focus on the stochastic Polyak step size, a step size strategy requiring very little fine-tuning of parameters. At the end of this theoretical part, we prove the convergence of stochastic gradient descent with stochastic Polyak step sizes. In the practical part, we first implement and compare the different step size strategies numerically using a small test problem to gain a better understanding about their characteristics. Finally, we use stochastic gradient descent with Polyak’s step size to solve a parameter identification problem of an ordinary diffential equation with uncertain initial conditions.

510 Mathematik
Zitieren
ISO 690KARAKOC, Dylan, 2023. Adaptive Step Sizes for Stochastic Gradient Descent [Bachelor thesis]. Konstanz: Universität Konstanz
BibTex
@mastersthesis{Karakoc2023Adapt-68032,
year={2023},
school={Universität Konstanz},
author={Karakoc, Dylan}
}
RDF
<rdf:RDF
xmlns:dcterms="http://purl.org/dc/terms/"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:bibo="http://purl.org/ontology/bibo/"
xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
xmlns:foaf="http://xmlns.com/foaf/0.1/"
xmlns:void="http://rdfs.org/ns/void#"
xmlns:xsd="http://www.w3.org/2001/XMLSchema#" >
<dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-11-03T07:18:46Z</dcterms:available>
<dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/68032/4/Karakoc_2-195qgktu8dk3w1.pdf"/>
<dc:contributor>Karakoc, Dylan</dc:contributor>
<void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
<dc:creator>Karakoc, Dylan</dc:creator>
<dcterms:abstract>In this thesis, we first lay some theoretical groundwork before motivating and discussing the stochastic gradient descent method along with its variations. We then analyze some popular step size strategies with a focus on the stochastic Polyak step
size, a step size strategy requiring very little fine-tuning of parameters. At the end of this theoretical part, we prove the convergence of stochastic gradient descent with stochastic Polyak step sizes. In the practical part, we first implement and compare the different step size strategies numerically using a small test problem to gain a better understanding about their characteristics. Finally, we use stochastic gradient descent with Polyak’s step size to solve a parameter identification problem of an ordinary diffential equation with uncertain initial conditions.</dcterms:abstract>
<bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/68032"/>
<foaf:homepage rdf:resource="http://localhost:8080/"/>
<dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/39"/>
<dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/68032/4/Karakoc_2-195qgktu8dk3w1.pdf"/>
<dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-11-03T07:18:46Z</dc:date>
<dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/39"/>
</rdf:RDF>