Publikation:

Adaptive Step Sizes for Stochastic Gradient Descent

Lade...
Vorschaubild

Dateien

Karakoc_2-195qgktu8dk3w1.pdf
Karakoc_2-195qgktu8dk3w1.pdfGröße: 1.05 MBDownloads: 99

Datum

2023

Autor:innen

Herausgeber:innen

Kontakt

ISSN der Zeitschrift

Electronic ISSN

ISBN

Bibliografische Daten

Verlag

Schriftenreihe

Auflagebezeichnung

DOI (zitierfähiger Link)
ArXiv-ID

Internationale Patentnummer

Link zur Lizenz

Angaben zur Forschungsförderung

Projekt

Open Access-Veröffentlichung
Open Access Green
Core Facility der Universität Konstanz

Gesperrt bis

Titel in einer weiteren Sprache

Publikationstyp
Bachelorarbeit
Publikationsstatus
Published

Erschienen in

Zusammenfassung

In this thesis, we first lay some theoretical groundwork before motivating and discussing the stochastic gradient descent method along with its variations. We then analyze some popular step size strategies with a focus on the stochastic Polyak step size, a step size strategy requiring very little fine-tuning of parameters. At the end of this theoretical part, we prove the convergence of stochastic gradient descent with stochastic Polyak step sizes. In the practical part, we first implement and compare the different step size strategies numerically using a small test problem to gain a better understanding about their characteristics. Finally, we use stochastic gradient descent with Polyak’s step size to solve a parameter identification problem of an ordinary diffential equation with uncertain initial conditions.

Zusammenfassung in einer weiteren Sprache

Fachgebiet (DDC)
510 Mathematik

Schlagwörter

Konferenz

Rezension
undefined / . - undefined, undefined

Forschungsvorhaben

Organisationseinheiten

Zeitschriftenheft

Zugehörige Datensätze in KOPS

Zitieren

ISO 690KARAKOC, Dylan, 2023. Adaptive Step Sizes for Stochastic Gradient Descent [Bachelor thesis]. Konstanz: Universität Konstanz
BibTex
@mastersthesis{Karakoc2023Adapt-68032,
  year={2023},
  title={Adaptive Step Sizes for Stochastic Gradient Descent},
  address={Konstanz},
  school={Universität Konstanz},
  author={Karakoc, Dylan}
}
RDF
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/68032">
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-11-03T07:18:46Z</dcterms:available>
    <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/68032/4/Karakoc_2-195qgktu8dk3w1.pdf"/>
    <dc:contributor>Karakoc, Dylan</dc:contributor>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:creator>Karakoc, Dylan</dc:creator>
    <dcterms:abstract>In this thesis, we first lay some theoretical groundwork before motivating and discussing the stochastic gradient descent method along with its variations. We then analyze some popular step size strategies with a focus on the stochastic Polyak step
size, a step size strategy requiring very little fine-tuning of parameters. At the end of this theoretical part, we prove the convergence of stochastic gradient descent with stochastic Polyak step sizes. In the practical part, we first implement and compare the different step size strategies numerically using a small test problem to gain a better understanding about their characteristics. Finally, we use stochastic gradient descent with Polyak’s step size to solve a parameter identification problem of an ordinary diffential equation with uncertain initial conditions.</dcterms:abstract>
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/68032"/>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/39"/>
    <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/68032/4/Karakoc_2-195qgktu8dk3w1.pdf"/>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-11-03T07:18:46Z</dc:date>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/39"/>
    <dcterms:rights rdf:resource="http://creativecommons.org/licenses/by-nc-sa/4.0/"/>
    <dcterms:issued>2023</dcterms:issued>
    <dcterms:title>Adaptive Step Sizes for Stochastic Gradient Descent</dcterms:title>
    <dc:rights>Attribution-NonCommercial-ShareAlike 4.0 International</dc:rights>
    <dc:language>eng</dc:language>
  </rdf:Description>
</rdf:RDF>

Interner Vermerk

xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter

Kontakt
URL der Originalveröffentl.

Prüfdatum der URL

Prüfungsdatum der Dissertation

Hochschulschriftenvermerk
Konstanz, Universität Konstanz, Bachelorarbeit, 2023
Finanzierungsart

Kommentar zur Publikation

Allianzlizenz
Corresponding Authors der Uni Konstanz vorhanden
Internationale Co-Autor:innen
Universitätsbibliographie
Ja
Begutachtet
Diese Publikation teilen