Robotic Painting with Region-Based Techniques

Abstract

Painting is an essential medium of expression and has been an art form practiced by humans since before recorded history. Despite today's society's heavy reliance on automation, the field of robotic painting is still underexplored, with many open questions remaining. Robotic painting offers many unique challenges, the solution of which would make painting more accessible and help to better understand the process of human painting. This dissertation describes the entire process of building a general-purpose robotic painting system called e-David, which can automatically realize paintings on a canvas. The main novelty is a region-based approach to painting, through which the machine is able to fill sections of the canvas with locally adapted brush techniques. First, the history of human and robotic painting is introduced, and robotic painting is placed into a broad context with respect to robotics in general. Then, the hardware component of the system is described, and the design goals are set. Given the hardware foundations, we discuss brush dynamics and the precise creation of single brushstrokes using brush calibration methods. Next, we discuss automatic writing as an application of precise brush control and as a showcase for the general-purpose nature of the developed machine setup. Moving on from single-stroke techniques, we introduce the titular region-based techniques: We propose to decompose images into regions that can be filled in with brush techniques that are adapted to semantics, local structure, and neighborhood relationships between shapes. For this purpose, we discuss several region-filling techniques, which are derived from the observation of human-created artworks. We also introduce geometric analysis methods to judge regions' paintability and find paintable subdivisions of complex shapes. Overall, the system imitates aspects of human painting while also including techniques specialized for robotic painting. A full robotic painting pipeline is established through the development of an abstraction procedure, which allows us to generate paintable geometry from input pixel images. An overview of e-David exhibitions, during which a mobile painting robot was shown to the public, is given. The thesis concludes with an overview of potential future research topics in the field of robotic painting.