Akademik Yayınlar

Optimal artificial neural network architecture design for modeling an industrial ethylene oxide plant

Optimum selection of input variables, number of hidden neurons and connections among the network elements deliver the best configuration of an ANN, usually resulting in reduced over-fitting and improved test performance. This study focuses on the development of a superstructure-oriented feedforward ANN design and training algorithm whose impacts are demonstrated on an industrial Ethylene Oxide (EO) plant for the prediction of product related variables. Proposed method brings about a mixed integer nonlinear programming problem (MINLP) to be solved, which takes the existence of inputs, neurons, and connections among the network elements into account by binary variables in addition to continuous weights of existing connections. Our investigations show that almost 85% of the ANN connections are removed compared to the fully connected ANN (FC-ANN) with 50% decrease in the number of inputs of the ANN. The modified ANN delivers a better prediction performance over FC-ANN, since FC-ANN suffers from over-fitting.

Uncertainty Propagation Based MINLP Approach for Artificial Neural Network Structure Reduction

The performance of artificial neural networks (ANNs) is highly influenced by the selection of input variables and the architecture defined by hyper parameters such as the number of neurons in the hidden layer and connections between network variables. Although there are some black-box and trial and error based studies in the literature to deal with these issues, it is fair to state that a rigorous and systematic method providing global and unique solution is still missing. Accordingly, in this study, a mixed integer nonlinear programming (MINLP) formulation is proposed to detect the best features and connections among the neural network elements while propagating parameter and output uncertainties for regression problems. The objective of the formulation is to minimize the covariance of the estimated parameters while by (i) detecting the ideal number of neurons, (ii) synthesizing the connection configuration between those neurons, inputs and outputs, and (iii) selecting optimum input variables in a multi variable data set to design and ensure identifiable ANN architectures. As a result, suggested approach provides a robust and optimal ANN architecture with tighter prediction bounds obtained from propagation of parameter uncertainty, and higher prediction accuracy compared to the traditional fully connected approach and other benchmarks. Furthermore, such a performance is obtained after elimination of approximately 85% and 90% of the connections, for two case studies respectively, compared to traditional ANN in addition to significant reduction in the input subset.

Superstructure Optimization of Dimethyl Ether Process

Integration of process flowsheet simulators and optimization algorithms is a prominent approach to address simultaneous design and optimization of processes, which is represented by a mixed integer nonlinear programming (MINLP) formulation. In this study, DWSIM, a free and rarely used simulator, is used as a black-box function for the evaluation in genetic algorithm in MATLAB. Proposed approach is implemented to a dimethyl ether process, calculating optimum processing conditions in addition to structural decision variables including the feedstock type, reactor, and separation unit selections. Results show that syngas has the major impact on the process economics and is significantly more economical feedstock although high number of additional processing units are required.

Architectural Design of Chemical and Biological Pathways through Parameter Sensitivity Oriented Mixed Integer Formulations

Chemical and biological pathways include high number of rate expressions with tunable parameters to be estimated from the experimental data. The lack of spatial and temporal data might result in statistically ill-defined inverse problem with multiple solutions in parameter estimation. Furthermore, the computational load and identifiability problems increase once the disturbances and measurement noises appear especially when the model architecture is large. A simultaneous approach to obtain the pathway architecture and corresponding parameters based on the parameter sensitivity values and their contribution to uncertainty is scarce. In this study, a mixed integer nonlinear programming (MINLP) formulation is developed for the automatized selection and estimation of the most/least sensitive parameters among a larger subset through simultaneous evaluation of local sensitivity expressions in addition to pathway model in the optimization problem. Unlike traditional parameter estimation problems, the model training performance is introduced as constraint which is pre-defined to balance the structural reduction and fitting performance. Although the approach calls for more computational load compared to traditional methods, due to binary variables to represent the existence of parameters, evaluation of higher number of equations during optimization, additional nonlinearity terms, the rigorous solution algorithm delivers a more robust model architecture with tightened prediction bounds. The problem formulation is flexible and can be further tailored to various applications.

The approach is implemented on a complex chemical/biological pathway whose ordinary differential equations, explicitly derived sensitivity expressions and architecture forming binary variable including formulations are introduced to various MINLP solvers using Python/PYOMO computation environment. The preliminary results show a 30% decrease in the prediction bounds after the selection of 75% of the available parameters with no significant decrease in model mean prediction performance. The preliminary result for a particular biochemical network is presented in Fig. 1.

The proposed methodology might require further mathematical operations to account for nonlinearities and nonconvexities for a better pathway formation. Some developments in terms of reformulations and approximations to handle nonlinearities and nonconvexities are still in progress.

A mixed integer optimization formulation for the reduction of complex systems based on uncertainty

Chemical reactions are typically represented by the mathematical models that comprise ordinary differential equations. Increasing the number of parameters and variables in the models produces significant computational complexity in some circumstances. Furthermore, the uncertainty of the parameters and the cumulative rise of impacts on computations diminish the model's prediction accuracy. The elimination of uncertainty is critical during the analysis and decision-making process.

In this study, it is aimed to reduce the amount of uncertainty and computational cost with mixed integer nonlinear programming integrated with forward sensitivity analysis. The problem is solved with Python/PYOMO by using various solvers (SCIP, BONMIN, etc.). In the objective function, equations describing the forward sensitivity analysis and nonlinear cumulative accumulation are included. The parameters that reduce the uncertainty of estimation without any significant change in the mean prediction value were determined and removed from the reaction network. In addition to that, a reduced reaction network structure was obtained as a result of parameter elimination. The mixed integer nonlinear optimization formulation is flexible and extensible for different applications and needs.

A Simultaneous Training and Input Selection Algorithm for Classification Problems Using Piecewise Approximations

Heart disease diagnosis using few measurements is a challenging an important task considering the increasing population. Artificial Neural Networks (ANNs) are promising mathematical architectures once the training is performed in an elegant manner to avoid theoretical challenges related to high nonlinearity, nonconvexity using few input variables to ensure generalization capability. This study shows the impact of the piecewise linear approximation of nonlinear functions in ANN architecture and training problem to benefit from the mixed integer linear problem formulation for the simultaneous input selection and training to obtain mixed integer programming-based ANN (MIP-ANN). Proposed formulation is further tailored through linking constraints to remove the connections from the eliminated inputs to favor parameter identifiability. A publicly available dataset is considered as a case study of whose results are also compared to traditional ANN with all inputs (FC-ANN) and a relatively more straightforward but common input selection method (SKB-ANN). The results provide a comparable performance despite significant reduction in the input space in addition to significant computational and theoretical advantages thanks to advanced formulation.

Superstructure Optimization of Dimethyl Ether Process

Environmental concerns and economic considerations make it an inevitable task to integrate new fuel alternatives in the energy markets. Dimethyl ether (DME) is a relatively cleaner fuel and can be produced with many pathways. Superstructure optimization-based process synthesis is an important approach for obtaining feasible and competitive process among high number of alternatives in addition to calculation of corresponding operating conditions. Superstructure optimization incorporates a simulation environment for the calculation of the process output and a mixed integer-nonlinear programming problem (MINLP) which include integer variables for the selection of units or streams and continuous variables for the temperature, pressures and many other variables. Such a simultaneous approach exploits the potential of mathematical models and advanced optimization algorithms as superstructure contains a wide range of processing units, which hinders the decision-making through traditional methods.

In the literature, many studies were conducted using commercial simulation environments. In this study, we used open-source DWSIM and ChemSep for the calculation of optimum DME process architecture and operating conditions using Genetic Algorithm which is a common evolutionary metaheuristic optimization. The evolutionary algorithms and black-box architecture of process simulations environments enables relatively practical solution of non-convex process design tasks without formulating rigorous mathematical programming problems. Despite sub-optimality, the ease of implementation would not only increase the attention on optimization-based design but also provide satisfactory process architecture thanks to relatively simpler process models. The optimization problem is comprehensive as raw materials, reactor dimensions, distillation pathways and associated operating conditions are included. Preliminary calculations show that significant economic impacts can be obtained through the method.