news-please : a Generic News Crawler and Extractor

Kurzanzeige
dc.contributor.authorHamborg, Felix
dc.contributor.authorMeuschke, Norman
dc.contributor.authorBreitinger, Corinna
dc.contributor.authorGipp, Bela
dc.date.accessioned2018-03-22T07:38:28Z
dc.date.available2018-03-22T07:38:28Z
dc.date.issued2017eng
dc.description.abstractThe amount of news published and read online has increased tremendously in recent years, making news data an interesting resource for many research dis-ciplines, such as the social sciences and linguistics. However, large scale col-lection of news data is cumbersome due to a lack of generic tools for crawling and extracting such data. We present news-please, a generic, multi-language, open-source crawler and extractor for news that works out-of-the-box for a large variety of news websites. Our system allows crawling arbitrary news websites and extracting the major elements of news articles on those websites, i.e., title, lead paragraph, main content, publication date, author, and main im-age. Compared to existing tools, news-please features full website extraction requiring only the root URL.eng
dc.description.versionpublishedeng
dc.identifier.urihttps://kops.uni-konstanz.de/handle/123456789/41887
dc.language.isoengeng
dc.subjectnews crawler, news extractor, scraper, information extractioneng
dc.subject.ddc004eng
dc.titlenews-please : a Generic News Crawler and Extractoreng
dc.typeINPROCEEDINGSeng
dspace.entity.typePublication
kops.citation.bibtex
@inproceedings{Hamborg2017newsp-41887,
  year={2017},
  title={news-please : a Generic News Crawler and Extractor},
  number={70},
  isbn={978-3-86488-117-6},
  issn={0938-8710},
  publisher={Verlag Werner Hülsbusch},
  address={Glückstadt},
  series={Schriften zur Informationswissenschaft},
  booktitle={Everything changes, everything stays the same : Understanding Information Spaces; Proceedings of the 15th International Symposium of Information Science (ISI 2017), Berlin, Germany, 13th-15th March 2017},
  pages={218--223},
  editor={Gäde, Maria},
  author={Hamborg, Felix and Meuschke, Norman and Breitinger, Corinna and Gipp, Bela}
}
kops.citation.iso690HAMBORG, Felix, Norman MEUSCHKE, Corinna BREITINGER, Bela GIPP, 2017. news-please : a Generic News Crawler and Extractor. 15th International Symposium of Information Science (ISI 2017). Berlin, Germany, 13. März 2017 - 15. März 2017. In: GÄDE, Maria, ed. and others. Everything changes, everything stays the same : Understanding Information Spaces; Proceedings of the 15th International Symposium of Information Science (ISI 2017), Berlin, Germany, 13th-15th March 2017. Glückstadt: Verlag Werner Hülsbusch, 2017, pp. 218-223. Schriften zur Informationswissenschaft. 70. ISSN 0938-8710. ISBN 978-3-86488-117-6deu
kops.citation.iso690HAMBORG, Felix, Norman MEUSCHKE, Corinna BREITINGER, Bela GIPP, 2017. news-please : a Generic News Crawler and Extractor. 15th International Symposium of Information Science (ISI 2017). Berlin, Germany, Mar 13, 2017 - Mar 15, 2017. In: GÄDE, Maria, ed. and others. Everything changes, everything stays the same : Understanding Information Spaces; Proceedings of the 15th International Symposium of Information Science (ISI 2017), Berlin, Germany, 13th-15th March 2017. Glückstadt: Verlag Werner Hülsbusch, 2017, pp. 218-223. Schriften zur Informationswissenschaft. 70. ISSN 0938-8710. ISBN 978-3-86488-117-6eng
kops.citation.rdf
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/41887">
    <dcterms:abstract xml:lang="eng">The amount of news published and read online has increased tremendously in recent years, making news data an interesting resource for many research dis-ciplines, such as the social sciences and linguistics. However, large scale col-lection of news data is cumbersome due to a lack of generic tools for crawling and extracting such data. We present news-please, a generic, multi-language, open-source crawler and extractor for news that works out-of-the-box for a large variety of news websites. Our system allows crawling arbitrary news websites and extracting the major elements of news articles on those websites, i.e., title, lead paragraph, main content, publication date, author, and main im-age. Compared to existing tools, news-please features full website extraction requiring only the root URL.</dcterms:abstract>
    <dc:creator>Gipp, Bela</dc:creator>
    <dc:contributor>Meuschke, Norman</dc:contributor>
    <dcterms:issued>2017</dcterms:issued>
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/41887"/>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2018-03-22T07:38:28Z</dc:date>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dcterms:title>news-please : a Generic News Crawler and Extractor</dcterms:title>
    <dc:creator>Breitinger, Corinna</dc:creator>
    <dc:contributor>Gipp, Bela</dc:contributor>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:creator>Hamborg, Felix</dc:creator>
    <dc:contributor>Hamborg, Felix</dc:contributor>
    <dc:contributor>Breitinger, Corinna</dc:contributor>
    <dc:creator>Meuschke, Norman</dc:creator>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dc:language>eng</dc:language>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2018-03-22T07:38:28Z</dcterms:available>
  </rdf:Description>
</rdf:RDF>
kops.conferencefield15th International Symposium of Information Science (ISI 2017), 13. März 2017 - 15. März 2017, Berlin, Germanydeu
kops.date.conferenceEnd2017-03-15eng
kops.date.conferenceStart2017-03-13eng
kops.flag.knbibliographytrue
kops.location.conferenceBerlin, Germanyeng
kops.sourcefieldGÄDE, Maria, ed. and others. <i>Everything changes, everything stays the same : Understanding Information Spaces; Proceedings of the 15th International Symposium of Information Science (ISI 2017), Berlin, Germany, 13th-15th March 2017</i>. Glückstadt: Verlag Werner Hülsbusch, 2017, pp. 218-223. Schriften zur Informationswissenschaft. 70. ISSN 0938-8710. ISBN 978-3-86488-117-6deu
kops.sourcefield.plainGÄDE, Maria, ed. and others. Everything changes, everything stays the same : Understanding Information Spaces; Proceedings of the 15th International Symposium of Information Science (ISI 2017), Berlin, Germany, 13th-15th March 2017. Glückstadt: Verlag Werner Hülsbusch, 2017, pp. 218-223. Schriften zur Informationswissenschaft. 70. ISSN 0938-8710. ISBN 978-3-86488-117-6deu
kops.sourcefield.plainGÄDE, Maria, ed. and others. Everything changes, everything stays the same : Understanding Information Spaces; Proceedings of the 15th International Symposium of Information Science (ISI 2017), Berlin, Germany, 13th-15th March 2017. Glückstadt: Verlag Werner Hülsbusch, 2017, pp. 218-223. Schriften zur Informationswissenschaft. 70. ISSN 0938-8710. ISBN 978-3-86488-117-6eng
kops.title.conference15th International Symposium of Information Science (ISI 2017)eng
relation.isAuthorOfPublicationb3446266-8107-4eb9-9221-75e81cca13df
relation.isAuthorOfPublicatione3f81adb-a670-4c4c-bade-6781b8f996b0
relation.isAuthorOfPublicationebdceabd-fdd9-44b2-b7bb-57ebea6b5574
relation.isAuthorOfPublication358ad52f-dab7-4582-bf8e-8adcf477a2d4
relation.isAuthorOfPublication.latestForDiscoveryb3446266-8107-4eb9-9221-75e81cca13df
source.bibliographicInfo.fromPage218eng
source.bibliographicInfo.seriesNumber70eng
source.bibliographicInfo.toPage223eng
source.contributor.editorGäde, Maria
source.flag.etalEditortrueeng
source.identifier.isbn978-3-86488-117-6eng
source.identifier.issn0938-8710eng
source.publisherVerlag Werner Hülsbuscheng
source.publisher.locationGlückstadteng
source.relation.ispartofseriesSchriften zur Informationswissenschafteng
source.titleEverything changes, everything stays the same : Understanding Information Spaces; Proceedings of the 15th International Symposium of Information Science (ISI 2017), Berlin, Germany, 13th-15th March 2017eng

Dateien

Sammlungen

Informatik und Informationswissenschaft: Publikationen