Download the full-sized PDF of Data Quality Assessment for Closed-Loop System Identification and Forecasting with Application to Soft SensorsDownload the full-sized PDF



Permanent link (DOI):


Export to: EndNote  |  Zotero  |  Mendeley


This file is in the following communities:

Graduate Studies and Research, Faculty of


This file is in the following collections:

Theses and Dissertations

Data Quality Assessment for Closed-Loop System Identification and Forecasting with Application to Soft Sensors Open Access


Other title
Monte Carlo simulations
sampling time
soft senor
time delay
model segmentation
heated tank
order-based conditions
closed-loop identification
data quality assessment
routine operating data
parameter-based conditions
Type of item
Degree grantor
University of Alberta
Author or creator
Shardt, Yuri
Supervisor and department
Huang, Biao (Chemical and Materials Engineering)
Examining committee member and department
Prasad, Vinnay (Chemical and Materials Engineering)
Chen, Tongwen (Electrical and Computer Engineering)
Shah, Sirish (Chemical and Materials Engineering)
Qin, S. Joe (Viterbi School of Engineering at University of Southern California)
Department of Chemical and Materials Engineering
Process Engineering
Date accepted
Graduation date
Doctor of Philosophy
Degree level
In many chemical plants, data historians store thousands of variables at fast sampling rates. Much of this collected data is routine operating data that could easily be used for system identification and forecasting, especially in the design of soft sensors. Currently, there is no framework for assessing the quality of these data sets. Therefore, this dissertation proposes a two-step method, consisting of data segmentation and data quality assessment, with application to soft sensor development. The first step, data segmentation, seeks to partition the extracted data set into regions that can be described using the same model. This step is necessary to avoid making the data set seem more informative than it truly is. Data segmentation is performed using a signal-entropy based approach for which the statistical properties were developed. Based on these results, it was proposed that monitoring the difference in entropy between the input and output signals for routine operating data is sufficient to partition the data set into its constituent models. Furthermore, it was shown that this difference is independent of the input signal properties under the assumption that the process is running in closed-loop without external excitation. The second step, data quality assessment, seeks to assess the actual data quality of each region. For data quality assessment for system identification, the condition number of the Fisher information matrix is shown to agree well with the developed and extended parameter- and order-based theoretical conditions for identification that included generalised pole-zero cancellations. For forecasting, data quality assessment, a Z-score based on the distribution of the measurement errors was proposed. Finally, in the process of testing this framework on a soft sensor system, it was discovered that the configuration of a closed-loop soft sensor can determine the success of the soft sensor development. Based on a detailed theoretical analysis of the soft-sensor system, it was shown that the presence of an integrator in the bias update term improves the tracking performance of the system. The individual steps as well as the complete framework were extensively tested using different simulation configurations and a pilot-scale, heated tank system.
Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.
Citation for previous publication
Yuri A.W. Shardt, Yu Zhao, Kwan Ho Lee, Xinyi Yu, Biao Huang, and Sirish Shah (2012). “Determining the State of a Process Control System: Current Trends and Future Challenges” in Canadian Journal of Chemical Engineering, vol. 90, no. 2, pp. 217–245Yuri A.W. Shardt and Biao Huang (2012). “Data Quality Assessment For System Identification: A Proposed New Framework” in the Proceedings of the Chemical Process Control Conference VII, Savannah, Georgia, United States of AmericaYuri A.W. Shardt and Biao Huang (2011), “Closed-Loop Identification Condition for ARMAX Models Using Routine Operating Data,” Automatica, vol. 47, no. 7, pp. 1534-1537Yuri A.W. Shardt and Biao Huang (2011), “Closed-Loop Identification with Routine Operating Data: Effect of Time Delay and Sampling Time,” Journal of Process Control, vol. 21, no. 7, pp. 997-1010Yuri A.W. Shardt and Biao Huang (2010), "Conditions for Identifiability Using Routine Operating Data for a First-Order ARX Process Regulated by a Lead-Lag Controller," in DYCOPS 2010, Leuven, BelgiumYuri A.W. Shardt and Biao Huang (2012), “Tuning a Soft Sensor's Bias Update Term. 1. The Open-Loop Case,” Industrial and Engineering Chemistry Research, vol. 51, no. 13, pp. 4958–4967Yuri A.W. Shardt and Biao Huang (2012), “Tuning a Soft Sensor's Bias Update Term. 2. The Closed-Loop Case,” Industrial and Engineering Chemistry Research, vol. 51, no. 13, pp. 4968–4981Yuri A.W. Shardt, Vinay Prasad, and Biao Huang (2011). “The Heated Tank Laboratory for Teaching Control to Undergraduates: A Structured Approach and Some Observations” at the 61st Canadian Chemical Engineering Conference, October 24th-26th, 2011, London, Ontario, Canada

File Details

Date Uploaded
Date Modified
Audit Status
Audits have not yet been run on this file.
File format: pdf (Portable Document Format)
Mime type: application/pdf
File size: 9679757
Last modified: 2015:10:12 13:24:37-06:00
Filename: Shardt_Yuri_Fall 2012.pdf
Original checksum: 3148fc6b7c685cfcca520fca351b36b8
Well formed: true
Valid: true
Status message: Too many fonts to report; some fonts omitted. Total fonts = 2216
File title: Acknowledgements
File title: Summary
File author: Yuri
Page count: 419
Activity of users you follow
User Activity Date