Reconfigurable Image Signal Processors for Nonlinear CMOS Image Sensors

• Author / Creator

  Ribeiro do Nascimento, Maikon

• Moore’s Law is dying, but this is breathing new life into system-on-chip (SoC) architectures. Industry leaders like Intel and Xilinx are investing in heterogeneous computing devices like the reconfigurable SoC, which integrates a field-programmable gate array (FPGA), a multi-core microprocessor (μP), and a variety of peripheral interfaces on the same silicon chip. Market drivers include autonomous vision systems, which benefit from the hard real-time capabilities of the FPGA, the operating system capabilities of the μP, and the sensor/networking capabilities of the interfaces. To these ends, this thesis offers a case study on the image signal processor (ISPr) of a nonlinear complementary metal-oxide semiconductor (CMOS) imaging system. Unlike a linear CMOS image sensor (CIS), a nonlinear CIS is able to image high/wide
  dynamic range scenes in single exposures at video rates, ideal features for outdoor applications involving motion. The thesis leverages previously-published image signal processing (ISPg) algorithms, identified clearly as background work, to develop novel digital circuit methods for fixed pattern noise correction, salt-and-pepper noise filtering, and histogram-based tone mapping.
  Beyond digital circuits for a specific CIS, the proposed digital circuit methods generate circuits for an arbitrary monotonic CIS, such as linear, log, or linlog, based on supplied parameters. Generated circuits are validated for a variety of parameters, including megapixel scenarios at standard video rates. They are
  shown to be very efficient, in terms of circuit complexity, maximum frequency, and power consumption, and have significant advantages over literature approaches, including ones that use application-specific integrated circuits and
  graphics processing units. As a stepping stone to exploiting the ISPr, for a nonlinear CIS in an autonomous vision system, the thesis also proposes a novel design flow, for a reconfigurable SoC, to make the FPGA the master of the imaging system, via interrupt service routines, running on the μP, and direct memory access circuits, embedded in the FPGA. Two approaches are investigated to do this: one favours open source tools, and the other performance.
  The work demonstrates local processing of high definition video in hard real time with results communicated over a web page on demand.

• Subjects / Keywords

  • isp
  • fpga
  • image signal processors
  • FPN
  • TMO

• Graduation date

  Spring 2020

• Type of Item

  Thesis

• Degree

  Doctor of Philosophy

• DOI

  https://doi.org/10.7939/r3-wdxs-7659

• License

  Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.